Systems and methods for multi-resolution image processing

ABSTRACT

Systems and methods for the processing of images over bandwidth-limited transmission interfaces, such as processing of high resolution video images over standard analog video interfaces, using multi-stream and/or multi-resolution analog methodology. The disclosed systems and methods may also be implemented to provide video resolutions across standard analog video interfaces in a manner that supports digital capture and imaging techniques to obtain useful information from digitally zoomed and enhanced video.

This patent application claims priority to copending U.S. Provisionalpatent application Ser. No. 60/456,294, filed Mar. 20, 2003, andentitled “Systems And Methods For Creation, Transmission, And Viewing OfMulti-Resolution Video” by Washington, the entire disclosure of which isincorporated herein by reference.

BACKGROUND OF THE INVENTION

The present invention relates generally to video systems, and in oneexemplary application to multi-image video systems.

Existing closed circuit television (“CCTV”) analog videotransmission-based surveillance installations typically employ ITU/ISONTSC (720×480) or PAL (720×576) standard video resolutions that arelimited by realizable system bandwidth and cost constraints associatedwith such installations. FIG. 1 depicts an example of such a prior artCCTV installation 100 including a standard resolution (i.e., NTSC orPAL) digital camera 102 that is connected to a multi-port personalcomputer-based digital video recorder (“DVR”) 104 through abandwidth-limited analog transmission interface 106 to provide astandard resolution displayed image 108.

Factors that limit the maximum bandwidth of such prior art installationsinclude signal attenuation associated with required maximum installationcabling length (typically up to 1000 feet), the type of cabling used(typically standard RG59 coax) and standard installation techniques thatcan result in non-optimal terminations. Increased costs associated withuse of improved cabling, improved installation techniques, and/or custominterface circuitry preclude their use to achieve increased bandwidthcapability in many situations. Transmission of higher resolution videoimagery may be accomplished using digital transmission technology, butrequires additional investments for new higher cost equipment, includingcompression and transport circuitry.

SUMMARY OF THE INVENTION

Disclosed herein are systems and methods that may be employed for theprocessing (e.g., creation, transmission and/or reconstruction) of adesired video scene (e.g., CCTV scene). In one embodiment, the disclosedsystems and methods may be employed for the analog transmission of avideo scene, including transmission of the desired scene over standardanalog interfaces (e.g., standard CCTV interfaces), and for the displayof the desired scene simultaneously in multiple resolutions (e.g., highand standard resolutions simultaneously), and/or for the display ofmultiple images of the desired scene on an analog display (e.g.conventional analog composite video monitor), using multi-stream (e.g.,dual-stream analog) methodology.

Advantageously, the disclosed systems and methods may be implemented inone embodiment to fulfill the need for higher-than-standard videoresolution imaging for video surveillance installations using existingdigital video equipment (e.g., using existing standard digital to analogencoders, existing coax connections, existing chipsets, existingstandard analog to digital de-encoders, etc.) with modified or newsoftware capable of implementing one or more features of the disclosedsystems and methods (e.g., in one exemplary embodiment by modifyingWindows-based DVR software on PC or Intel-based DVR systems). Using thedisclosed systems and methods CCTV video resolutions may be providedthat support and take advantage of digital capture and imagingtechniques to obtain useful information from digitally zoomed andenhanced video.

In another embodiment, the disclosed systems and methods may beimplemented to generate multiple camera views from a relatively higherresolution image source (e.g., high resolution image sensor, imagestorage device containing high resolution image data, etc.) fortransmission and display over relatively lower resolution video mediacomponents (e.g., standard video media such as across a standard analogcomposite video interface for display on a standard analog compositevideo display monitor). In such an embodiment, multiple imageinformation streams may be integrated into a standard definition videostream as embedded camera views and/or may be sent as individual videostreams, simultaneously over additional video media. In one exemplaryembodiment, a single camera or video image storage device may beemployed to virtualize and emulate multiple image sources (e.g.,multiple cameras) with individual viewing and control capabilities.

As used herein, “high resolution” may be characterized as a videoresolution that is greater than standard NTSC or PAL resolutions.Therefore, in one embodiment the disclosed systems and methods may beimplemented to provide a resolution greater than standard NTSC andstandard PAL resolutions, or greater than 720×576 pixels (414,720pixels, or greater), across a standard composite video analog interfacesuch as standard coaxial cable. Examples of some common high resolutiondimensions include, but are not limited to: 800×600, 852×640, 1024×768,1280×720, 1280×960, 1280×1024, 1440×1050, 1440×1080, 1600×1200,1920×1080, and 2048×2048. In another embodiment, the disclosed systemsand methods may be implemented to provide a resolution greater thanabout 800×600 pixels (i.e., 480,000 pixels), alternatively to provide aresolution greater than about 1024×768 pixels, and further alternativelyto provide HDTV resolutions of 1280×720 or 1920×1080 across a standardcomposite video analog interface such as standard coaxial cable.Examples of high definition standards of 800×600 or greater that may beso implemented in certain embodiments of the disclosed systems andmethods include, but are not limited to, consumer and PC-based digitalimaging standards such as SVGA, XGA, SXGA, etc. It will be understoodthat the forgoing examples are representative of exemplary embodimentsonly and that the disclosed systems and methods may be implemented toprovide enhanced resolution that is greater than the native or standardresolution capability of a given video system, regardless of theparticular combination of image source resolution and type of interface.

In another embodiment, high resolution image source/s (e.g., 1024×768pixel range or greater) may be processed in a fashion that generatesmultiple output video streams, that may be used to provide separateviewing capabilities. This may be accomplished, for example, by takingthe video source output (e.g., image sensor based video output) whichmay be a stream of video ‘frames’, and generating (either physically orlogically) multiple video output streams to individual processing units.These video output streams may all be identical, or alternatively may besamplings of different visual regions of the output video frames of theimage source. It will be understood that the actual visual content ofeach video stream may vary according to the needs or desiredcharacteristics for a given application, and is independent of theimplementation of one or more features of the disclosed systems andmethods.

In the practice of the disclosed systems and methods, examples ofadvantages that may be realized (e.g., alone or in any combination)using the disclosed systems and methods to provide higher resolutionvideo and/or multiple-resolutions for video surveillance or other videosignal processing applications include, but are not limited to, thefollowing. Higher resolution and multiple resolution video of thedisclosed systems and methods may be implemented to allow for areduction in the number of cameras required to cover an area ofinterest. In one example, use of HDTV resolutions may allow for areduction in the number of cameras (e.g., of a video surveillancesystem) by a ratio approaching or equal to 4:1 since the HDTV horizontalresolution is roughly 4 times that of standard definition cameras.Stored content associated with higher resolution video of the disclosedsystems and methods may also be taken advantage of to provide moreuseful information. For example, in most cases involving the occurrenceof an event of interest (e.g., occurrence of a security event such astheft, break-in, assault, etc.), cameras are often not zoomed into thecorrect space or given spatial portion of the video image to capture themost relevant or detailed data associated with the event. However,higher resolution information captured using the disclosed systems andmethods may be used to provide substantially increased detail of a givenportion of a video image and to provide greater information as to whatwas occurring at the time around an event of interest, even when thevideo camera is not zoomed into that give portion of the video image.Multiple resolution images of the disclosed systems and methods may alsobe used to provide the capability to maintain an overall awareness of ageneral situation while allowing simultaneous focus/zoom on a specificarea/s of activity or interest.

In one embodiment, a multi-resolution video stream may be generated overa standard closed circuit television interface utilizing a highresolution sensor. The high resolution images from this sensor, may bescaled down to standard NTSC or PAL resolution and transmitted at aframe rate slower than that normally associated with NTSC or PAL framerates (e.g., 25 to 30 frames per second). Frame periods not occupied bythe scaled video images may then be used to transmit a tiled version ofthe full resolution image along with embedded information that allowsthe full high resolution image to be reconstructed without visibleartifacts. Image reconstruction may be performed by a personal computer,processor-based DVR, or other suitable processor or dedicated hardware,and the resultant high resolution image made available for display alongwith the lower resolution image on a high resolution monitor and/or madeavailable for digital storage. Using the embedded information alsoallows transmission of non-tiled versions of the original highresolution image intermixed with the tiled information. Examples wouldbe specific areas of the original high resolution image that have beenscaled, the entire original image scaled to a standard definition size,etc. In this way both a blend of lower frame rate high resolution imagesand higher frame rate standard resolution images may be sent across asingle interface

In such an embodiment, the disclosed systems and methods may beimplemented in a manner compatible with reduced frame ratecharacteristics of existing CCTV video surveillance industry technology.In this regard, typical digital video recorders provide a capability ofless than 30 frames/sec for video storage or display, with the averageframe rates utilized by the industry being in the range of from about 3to about 8 frames/sec. In the video surveillance industry, this framerate has been considered to provide an acceptable tradeoff betweensituational awareness, video storage size, cost, and processing powerrequirements. Therefore, high resolution images at these lower framerates may be acceptable, and/or high resolution images at even lowerframe rates may suffice for some situations where lower resolution(e.g., standard resolution images) are simultaneously available athigher frame rates. Although higher frame rates may be desirable forsome applications (e.g., such as trying to capture illegalsleight-of-hand techniques in gambling or theft situations), it has beenestimated that even these situations require only from about 15 to about18 frames/sec. In this regard, the disclosed systems and methods may beimplemented in one exemplary embodiment to provide standard resolutionimages within the range of from about 15 to about 30 frames/sec whilesimultaneously sending tiled images for subsequent reconstruction.

In another embodiment, the disclosed systems and methods may beimplemented to take advantage of the unused bandwidth associated withdiscarded or “dropped” frames of video that occur, for example, when agiven frame rate (e.g., 30 frames/sec) is available for transmission,but only a portion (e.g., about 2 to 7.5 frames/sec) of the given framerate is stored and/or displayed. This unused bandwidth may be utilizedto transmit higher resolution images, multiple resolution images,combinations thereof, etc.

In various embodiments of the disclosed systems and methods, a number ofexemplary features may be advantageously implemented, alone or incombination, to create, transmit, and/or view video scenes. Examples ofsuch exemplary features include, but are not limited to, creation of amultiple resolution video stream compatible with standard CCTVinterfaces; transmission of a multiple resolution video streamcompatible with standard CCTV interfaces; utilization of patterns withinunused portions of a video frame for proper reconstruction of higherresolution images to reduce or eliminate visual artifacts; andutilization of patterns within unused portions of a video frame formarking of image segments as to their original spatial relationship andsubsequent detection and reaction to these markings.

In the practice of the disclosed systems and methods, one or moreportions of an image produced by an image source (e.g., image sensor,image storage device, etc.) may be zoomed before or after transmissionacross an analog interface. For example, in one embodiment one or morezoomed portions of an image received from an image source may begenerated prior to transmission in real time across an analog interfacefor analog display with no further processing, i.e., as zoomed image/sproduced and then transmitted as part of analog video signals across ananalog interface by multi-stream image processing circuitry of a camera.In another embodiment, one or more portions of an image may be digitallyprocessed and zoomed in real time after receipt of the image from acrossan analog interface (e.g., after transmission of tiled images across ananalog interface and reconstruction of a high resolution image therefromin a PC-based DVR). In yet another embodiment, image information may befirst stored after receipt across an analog interface (e.g., with orwithout further digital processing to reconstruct a high resolution orother type image from multiple tiles), and then retrieved and furtherdigitally processed as appropriate to produce one or more desired zoomedimages (e.g., in a PC-based DVR or using any other suitable type of DVRor circuitry suitable for this purpose). The term “zoomed” herein refersto the spatial scaling of all or part of an original image. In thisregard, the scaling factor may range in value from less than 1(reduction) and into positive factors greater than 1 (enlargement).

In one respect, disclosed herein is a method of providing multipleimages for transmission across an analog interface, including: receivingat least one digital image data input stream, the digital image datainput stream containing digital image information; creating at least twodigital image data streams from the at least one digital data inputstream, each of the at least two digital image data streams including atleast a portion of the digital image information; merging the at leasttwo digital image data streams into a common digital image data outputstream; converting the common digital image data output stream into ananalog image output stream; and providing the analog output image streamfor transmission across the analog interface.

In another respect, disclosed herein is a method of providing multipleimages for transmission across an analog interface, including: receivingat least one digital image data input stream, the digital image datainput stream containing digital image information; creating at least twodigital image data streams from the at least one digital data inputstream, each of the at least two digital image data streams including atleast a portion of the digital image information, and wherein a firstone of the at least two digital image data streams includes a firstimage may have a first resolution, and wherein a second one of the atleast two digital image data streams includes a second image may have asecond resolution; converting a first one of the at least two digitalimage data streams into a first analog image output stream, andconverting a second one of the at least two digital image data streamsinto a second analog image output stream; communicating the first analogoutput image stream across a first analog interface, and communicatingthe second analog output image stream across a second analog interface;receiving the first analog output image stream from across the firstanalog interface, the first analog output image stream including thefirst image; receiving the second analog output image stream from acrossthe second analog interface, the second analog output image streamincluding the second image; displaying the first image of the firstanalog output image stream on a first analog display device; anddisplaying the second image of the second analog output image stream ona second analog display device.

In another respect, disclosed herein is a method of processing digitalimage data, including: providing the digital image data, the digitalimage data including a full image; processing the digital image data ina first processing operation to create first processed image dataincluding a first image; processing the digital image data in a secondprocessing operation to create second processed image data including asecond image; merging the first and second processed image data into acommon merged data stream; converting the merged data stream into acomposite analog video stream containing the first and second images;and providing the composite analog video stream for communication acrossan analog interface. The first processed image data may have an imageresolution that is different from an image resolution of the secondprocessed image data, or the first processed image data may include adifferent portion of the digital image data than the second processedimage data, or a combination thereof.

In another respect, disclosed herein is multiple resolution imagecreation circuitry configured to receive at least one digital image datainput stream containing digital information, the multiple resolutionimage creation circuitry including: multi-resolution image processingcircuitry configured to: create at least two digital image data streamsfrom the at least one digital data input stream, each of the at leasttwo digital image data streams including at least a portion of thedigital image information, merge the at least two digital image datastreams into a common digital image data output stream; and conversioncircuitry coupled to the multi-resolution image processing circuitry andconfigured to: receive the common digital image data output stream fromthe multi-resolution image processing circuitry, convert the commondigital image data output stream into an analog image output stream, andprovide the analog output image stream for transmission across an analoginterface. A multiple resolution image creation and display system mayinclude the multiple resolution image creation circuitry, and mayfurther include an analog display device coupled to the analog interfacethat is configured to receive the analog video signal from across theanalog interface and to simultaneously display the first and secondimages of the analog video signal.

In another respect, disclosed herein is multiple resolution imagecreation and display system, including multi-resolution image processingcircuitry, conversion circuitry, and first and second display devices.The multi-resolution image processing circuitry may be configured to:receive at least one digital image data input stream, the digital imagedata input stream containing digital image information, create at leasttwo digital image data streams from the at least one digital data inputstream, each of the at least two digital image data streams including atleast a portion of the digital image information, and wherein a firstone of the at least two digital image data streams includes a firstimage may have a first resolution, and wherein a second one of the atleast two digital image data streams includes a second image may have asecond resolution. The conversion circuitry may be coupled to themulti-resolution image processing circuitry and be configured to:convert a first one of the at least two digital image data streams intoa first analog image output stream, and convert a second one of the atleast two digital image data streams into a second analog image outputstream, communicate the first analog output image stream across a firstanalog interface, and communicate the second analog output image streamacross a second analog interface. The first display device may becoupled to the first analog interface and may be configured to receivethe first analog output image stream including the first image fromacross the first analog interface, and to display the first image. Thesecond display device may be coupled to the second analog interface andmay be configured to receive the second analog output image streamincluding the second image from across the second analog interface, andto display the second image.

In another respect, disclosed herein is multiple resolution imagecreation circuitry for processing digital image data including a fullimage, the multiple resolution image creation circuitry includingmulti-resolution image processing circuitry, and conversion circuitry.The multi-resolution image processing circuitry may be configured to:process the digital image data in a first processing operation to createfirst processed image data including a first image, process the digitalimage data in a second processing operation to create second processedimage data including a second image, and merge the first and secondprocessed image data into a common merged data stream. The conversioncircuitry may be configured to: convert the merged data stream into acomposite analog video stream containing the first and second images,and provide the composite analog video stream for communication acrossan analog interface. The first processed image data may have an imageresolution that is different from an image resolution of the secondprocessed image data, or the first processed image data may include adifferent portion of the digital image data than the second processedimage data, or a combination thereof. A multiple resolution imagecreation and display system may include the multiple resolution imagecreation circuitry, and may further include an analog display devicecoupled to the analog interface and configured to: receive the compositeanalog video stream containing the first and second images from acrossthe interface; and simultaneously display the first and second imagescontained in the composite analog video stream.

In another respect, disclosed herein is an image processing and displaysystem including multiple resolution image creation circuitry coupled toat least one analog display device by at least one analog interface.

In another respect, disclosed herein is a method of providing multipleimages for transmission across a digital interface, including: receivingat least one digital image data input stream, the digital image datainput stream containing digital image information; creating at least twodigital image data streams from the at least one digital data inputstream, each of the at least two digital image data streams including atleast a portion of the digital image information; merging the at leasttwo digital image data streams into a common digital image data outputstream; providing the common digital image data output stream fortransmission across the digital interface.

In another respect, disclosed herein is multiple resolution imagecreation circuitry configured to receive at least one digital image datainput stream containing digital information, the multiple resolutionimage creation circuitry including multi-resolution image processingcircuitry configured to: create at least two digital image data streamsfrom the at least one digital data input stream, each of the at leasttwo digital image data streams including at least a portion of thedigital image information, merge the at least two digital image datastreams into a common digital image data output stream; and provide thecommon digital image data output stream for transmission across adigital interface. A multiple resolution image creation and displaysystem may include the multiple resolution image creation circuitry, andmay further include a digital processing device coupled to the digitalinterface and configured to receive the common digital image data outputstream from across the digital interface and to store the common digitalimage data output stream, to display one or more images contained in thecommon digital image data output stream, or a combination thereof.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a simplified schematic of a prior art standard resolution CCTVvideo surveillance system.

FIG. 2 is a simplified schematic of a multiple resolution video systemaccording to one embodiment of the disclosed systems and methods.

FIG. 3A is a simplified schematic of a video system according to oneembodiment of the disclosed systems and methods.

FIG. 3B is a simplified schematic of a video system according to oneembodiment of the disclosed systems and methods.

FIG. 4A is a representation of four individual image tiles embedded invideo frames as they may be oriented for reassembly into a reconstructedimage according to one embodiment of the disclosed systems and methods.

FIG. 4B illustrates correlation of an alignment pattern according to oneembodiment of the disclosed systems and methods.

FIG. 4C illustrates blending of tile overlap areas according to oneembodiment of the disclosed systems and methods.

FIG. 4D represents a reconstructed image according to one embodiment ofthe disclosed systems and methods.

FIG. 5A-5B represents selectable extraction of higher rate highresolution image area out of a lower rate high resolution imageaccording to one embodiment of the disclosed systems and methods.

FIG. 5C-5D represents selectable extraction of high resolution imagearea out of a larger standard resolution image according to oneembodiment of the disclosed systems and methods.

FIG. 6A represents an non-zoomed stored image taken by a videosurveillance camera.

FIG. 6B represents a digitally zoomed image of the non-zoomed storedimage of FIG. 6A obtained when starting with a standard resolutionimage.

FIG. 6C represents a digitally zoomed image of the non-zoomed storedimage of FIG. 6A obtained when starting with a high resolution imageaccording to one embodiment of the disclosed systems and methods.

FIG. 7 represents selectable extraction, scaling, and display of imageareas out of a higher resolution image according to one embodiment ofthe disclosed systems and methods.

FIG. 8A is a diagram showing image data logic flow according to oneembodiment of the disclosed systems and methods.

FIG. 8B is a diagram showing image data logic flow according to oneembodiment of the disclosed systems and methods.

FIG. 9 illustrates a de-interlaced active frame according to oneembodiment of the disclosed systems and methods.

FIG. 10 illustrates placement of a tile identifier pattern in a verticalframe according to one embodiment of the disclosed systems and methods.

FIG. 11A-11D illustrates tile identifier patterns according to oneembodiment of the disclosed systems and methods.

FIG. 12 illustrates display of multiple images according to oneembodiment of the disclosed systems and methods.

FIG. 13 illustrates image reconstruction methodology according to oneembodiment of the disclosed systems and methods.

FIG. 14 illustrated selection of a variable sized subset of a largerresolution image and positioning of the selection area within the largerimage in relationship to external pan and tilt commands.

DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS

The disclosed systems and methods may be implemented to allow imageinformation that originates from one or more image sources to beprocessed and provided across a transmission interface in multipleresolutions and/or multiple streams in a manner that allows forreconstruction, storage and/or display as one or more images on one ormore suitable analog and/or digital video display devices. In thisregard, it will be understood that the actual image content, format,and/or spectral characteristics (e.g., visual image, infrared image,radar image, etc.) of image streams processed and transmitted accordingto the disclosed systems and methods may vary according to the needs ordesired characteristics for a given application, and is independent ofthe implementation of one or more features of the disclosed systems andmethods.

In one embodiment, the disclosed systems and methods may be employed forreceiving image information having a native resolution and frame ratefrom one or more image source/s, and for processing and providing theimage information across an analog interface for display in digitalformat as one or more images of native or less than native resolutionand/or of native or less than native frame rate (e.g., as one or morenative high resolution image/s, as one or more images of native and/orless than native frame rate, as one images of native or less than nativeresolution, as one or more zoomed or non-zoomed images, or as anycombination thereof). In this regard, a single image may be displayed ormultiple images may be simultaneously displayed, e.g., two or moreimages of multiple resolution, frame rate, and/or zoom scale may besimultaneously displayed on a single digital display device (e.g.,computer monitor) or on multiple digital display devices. For example, adesired scene may be transmitted over standard analog interface (e.g.,standard CCTV interface), and displayed simultaneously in multipleresolutions (e.g., windows containing high and standard resolutionsimultaneously).

In another embodiment, the disclosed systems and methods may be employedfor receiving image information having a native resolution and framerate from one or more image source/s, and for processing and providingthe image information across an analog interface for display in analogformat as one or more images (e.g., as one or more zoomed or unzoomedimages, or a combination thereof). In this regard, a signal image may bedisplayed or multiple images may be simultaneously displayed on a singleanalog display device (e.g. conventional analog composite video monitor)or on multiple analog display devices. For example, a desired scene maybe transmitted over standard analog interface (e.g., standard CCTVinterface), and displayed simultaneously in multiple zoom scales (e.g.,unzoomed and zoomed windows displayed simultaneously).

Transmission Across Analog Interface for Display on Digital DisplayDevice

In one embodiment, the disclosed systems and methods may be implementedwith multiple image creation circuitry that may be coupled to create andprovide multiple image information (e.g., multiple-resolution and/ormultiple-stream image information) via analog signals to multiple imageprocessing circuitry for image storage, image display, and/or furtherimage processing. As an example, FIG. 2 illustrates one exemplaryembodiment of a video system 200 having multiple resolution imagecapability that may be implemented in the practice of the disclosedsystems and methods, for example, to create and providemultiple-resolution image transmission capability over a conventionalanalog transmission interface 206, e.g., having a frequency limit of 4-5MHz. As shown in FIG. 2, video system 200 may include multiple streamimage creation circuitry in the form of an enhanced resolution digitalcamera 202 that includes multiple stream image processing components(224, 225, 226, 228, 230, 232), the features of which may be implementedusing any suitable hardware and/or software configuration (e.g., digitalsignal processor (“DSP”), application specific integrated circuit, fieldprogrammable gate array, combinations thereof, etc.). Also shown in FIG.2 is multiple stream image receiving circuitry in the form of a PC-basedDVR 204 that includes image reconstruction and multiple streamprocessing/storage capabilities. As shown in FIG. 2, video system 200may be implemented in one embodiment to provide a multiple resolutionimage display on a standard PC monitor 208 (e.g., simultaneous displayof both standard resolution image 209 and high resolution image 211).

Although FIG. 2 illustrates a multiple resolution video system 200capable of providing multiple resolution image capability (e.g.,standard and high resolutions in this exemplary embodiment), it will beunderstood that the disclosed systems and methods may be alternativelyimplemented to provide a video system capable of providing multiple(e.g., two or more, three or more, etc.) resolutions and/or that themultiple image resolutions may be selected to be any combination ofdesired resolutions, e.g. whether the multiple resolutions are selectedto each be high resolutions, each to be standard resolutions, or whethera combination of such resolutions is selected.

Furthermore, it will be understood that it is not necessary that thedisclosed systems and methods be practiced with two or more imageresolutions that are not standard and/or high resolution images asdefined herein. In this regard, any two or more multiple resolutions ofdesired different resolutions (i.e., of lower and higher resolutionrelative to each other) may be selected, including selected resolutionsthat are neither standard or high resolution. In this regard, a multiplestream image system having multiple resolution capability may beemployed, for example, in the embodiment described with relation to FIG.2 for creation, transmission and optional display of standard and highresolution images, it being understood that this embodiment is exemplaryonly and that other embodiments of the disclosed systems and methods maybe configured for the creation, transmission and optional display ofthree or more images of differing resolutions, and/or for the creation,transmission and optional display of two or more images with differingresolutions that are not necessarily either standard and/or highresolution images. In addition, it will also be understood that thedisclosed systems and methods need not be practiced to process andtransmit multiple image resolutions, but instead may be implemented toprocess and transmit a single resolution using multiple streams.

In FIG. 2, enhanced resolution digital camera 202 is shown coupled toPC-based DVR 204 by bandwidth-limited analog interface 206 (e.g.,similar to the bandwidth-limited analog interface 106 of FIG. 1). Asused herein, the term “bandwidth limited interface” is used to describea signal transmission format (e.g., standard composite video analogtransmission interface) that has the characteristic of limitedtransmission capacity relative to the transmission capacity required totransmit a given image signal (e.g., high resolution image signal). Oneexample of a bandwidth limited interface is a typical coaxialcable-based analog video transmission format that is limited toapproximately 485 TV lines of resolution due to the following factors:

-   -   Bandwidth limitations associated with transmission of a level        based signal (such as analog video) over coaxial cable which        will roll off the higher frequencies as the length of the cable        increases.    -   Encoding of the luminance and chrominance into a single        composite video signal which requires frequency attenuation of        the luminance data in order for it to not be interpreted as        chrominance information.    -   Limitations built into existing frame grabber interfaces which        are designed to work with the expected video bandwidths.

It will be understood that a standard composite video analogtransmission interface is only one example of a bandwidth limitedinterface. Furthermore, although a bandwidth-limited interface isillustrated in FIG. 2, it will be understood that the disclosed systemsand methods may be implemented to provide images over any other type ofinterface/transmission media suitable for analog and/or digital signaltransmission, including interfaces that are not bandwidth-limitedrelative to the desired signal to be transmitted. Other examples ofinterfaces with which the disclosed systems and methods may beadvantageously employed to transmit a desired image signal usingmulti-stream methodology include, but are not limited to, fiber opticinterface formats, wireless (radio frequency) interface formats, serialdigital formats, etc. In this regard, examples of other types ofsuitable interfaces include, but are not limited to, conversion todigital video for transmission over fiber, conversion to the RF domainfor transmission over coaxial cable or direct wireless transmission,etc. It will thus be understood that any type of signal may betransmitted (in bandwidth or in non-bandwidth limited format relative tothe interface), and that additional processing or conversion may beemployed as long as the transmission contains multi-stream informationin accordance with the systems and methods disclosed herein. In the casewhere the interface is non-bandwidth limited the techniques describedherein may be usefully employed, e.g., to reduce the data bandwidth inorder to preserve other system resources such as hard drive storage.

As illustrated for the exemplary embodiment of FIG. 2, digital camera202 includes optics 220 and an image source in the form of video orimage sensor 222, in this exemplary embodiment a high resolution videosensor having a standard HDTV resolution of 1280×720 (as opposed to thestandard 720×480 or 720×576 resolution of the video sensor of prior artdigital camera 102 of FIG. 1). Although a HDTV high resolution sensor isillustrated and described in relation to the exemplary embodiment ofFIG. 2, it will be understood with benefit of this disclosure that adigital camera may be provided with any other type and/or resolution ofhigh resolution or standard resolution image sensor or combination ofsuch sensors that are suitable for generating image information havingcharacteristics described further herein. Examples of suitable imagesensors include, but are not limited to, image sensors havingresolutions of 1920×1080, 1280×1024, 2048×2048, etc. Suitable types ofimage sensors include, but are not limited to, CCD, CMOS, ultraviolet,near infrared, infrared, etc.

Although video system 200 of the exemplary embodiment of FIG. 2 isconfigured having a camera with an image source in the form of an imagesensor, it will be understood that a multiple resolution video systemmay be configured in the form of a camera or in a non-cameraconfiguration with any one or more types of image sources that aresuitable for generating image information in any picture format, e.g.,including image sensors, storage devices capable of providing previouslygenerated and stored image information, high frame rate images generatedfrom RADAR or laser mapping, etc.). Furthermore, an image source may bea digital image source, or may be an analog image source producing ananalog signal that is converted to digital signal via digital to analog(“DAC”) conversion. In one embodiment, the disclosed systems and methodsmay be implemented to simultaneously receive, process and interleaveimage signals from two more image sources, e.g., image signalsoriginating from wide-angle view and narrow angle view image sensors,image signals from IR and visible spectrum image sources, image signalsfrom two image storage devices, etc.

As shown in FIG. 2, camera 202 may be configured with conditioningcircuitry 221 that is capable of converting information from sensor 222into video information having a desired digital video form. In thisregard, conditioning circuitry 221 may be any circuitry suitable forconverting raw sensor information into desired digital video form (e.g.,standard format digital video form) including, but not limited tocircuitry capable of converting CCD or CMOS sensor information (e.g.,the bayer pattern red-green-blue (“RGB”), or alternatelycyan-magenta-yellow (“CMY”)) into planar RGB or CMY. Other possibletasks which may be performed by conditioning circuitry 221 include, butare not limited to, the control of an image sensor to produce the propersignal levels (exposure control), providing the correct synchronizationof sensor control signals, and other conditioning aspects such asaperture correction (i.e. high frequency boost), white balance, colorcorrection, demosaicing, dynamic range processing, dynamic rangecompression, formatting, etc. Color space conversion such as RGB or CMYto 4:4:4, 4:2:2 or 4:2:0 format YCrCb may also be performed by circuitry221. In one exemplary embodiment, conditioning related processing may beperformed via embedded digital logic or via algorithms running on a DSPor any other suitable processing element. It will be understood thattype/s and combination/s of types of conditioning may vary and may beperformed by circuitry 221 as desirable to fit the characteristics of agiven application. It is also possible that no conditioning circuitrymay be present, for example, in the case of a digital video signalreceived from a video source in the desired signal form (e.g., receivedin the form of a standard format digital video signal).

As illustrated for the exemplary embodiment of FIG. 2, multiple streamimage creation circuitry in the form of camera 202 may be configuredwith multi-stream image processing circuitry 270 that includes Framestore circuitry 224 that stores the higher resolution frame, one or moreinstances of Window circuitry 225 that is configured to extract aselected portion of the higher resolution frame to form a full orwindowed partial image, one or more instances of Image Scaler circuitry226 that scales the higher resolution image down to a NTSC, PAL, SECAM,etc. compatible format or scales a smaller section of the higherresolution frame up to NTSC/PAL compatible formats for output over oneof the output interfaces 206, Image Deconstruction circuitry 228 thatsegments or “Tiles” the high resolution image into multiple tilesegments (e.g., four pieces), Alignment Data Insert circuitry 230 thatinserts alignment data into the unused lines of the tiled image, andImage Mux circuitry 232 that selects the information from separatesources (e.g., either the high resolution image scaled down to NTSC/PALresolutions, a high resolution tile, or zoomed segment of a highresolution image) and transmits it in a multiple stream (e.g., dualstream) format. In this regard, it will be understood that multipleinstances of window circuitry 225 and image scaler circuitry 226 may beprovided in order to implement separate image data paths that may bemultiplexed for transmission in a manner as will be described furtherherein, for example, in relation to FIG. 8A. Although not illustrated,it is also possible to have multiple instances of other circuitry withinmulti-stream image processing circuitry 270 (e.g., circuitry 228,circuitry 230, etc.) as necessary or desired to meet the requirements ofa given application.

Also shown in FIG. 2 is optional supplemental image processing circuitry223 that may be implemented as shown for purposes of implementingalgorithms such as facial or target recognition and advanced motiondetection algorithms. In such an exemplary embodiment, these types ofalgorithms may be provided with access to full frame rate and fullresolution video in it's highest quality form, e.g., before anychrominance downsampling to 4:2:2 and/or any degradation going to theanalog domain and back again. Such an implementation may be employed asan alternative or in addition to implementation of such algorithms inmultiple stream image receiving circuitry (e.g., such as PC-based DVR204) after transmission of analog signals across interface(s) 206.

In the illustrated embodiment, optional supplemental image processingcircuitry 223 may be any circuitry suitable for receiving higherresolution frames from a suitable image source, in this caseconditioning circuitry 221, and for implementing one or moresupplemental image processing algorithms/circuitry that is sensitive topixel resolution (i.e., sensitive to the number of pixels in an image)prior to providing frames to Frame store circuitry 224. In this regard,pixel resolution-sensitive algorithms/circuitry refers to anyalgorithms/circuitry that is affected by degradation or loss of pixelsfrom an original image (e.g., due to image processing in multi-streamimage processing circuitry 270 or other processing that results in lossof pixels). Specific examples of pixel resolution-sensitivealgorithms/circuitry include, but are not limited to, facial, licenseplate, or target recognition algorithms and circuitry used to implementsame; motion detection algorithms and circuitry used to implement same;etc. In this regard, increased pixel resolution may facilitaterecognition of movement, features, etc. within an image prior to furtherprocessing. Further, increased frame rate of an original image (e.g.prior to processing that reduces frame rate) also may benefit operationof such algorithms/circuitry. Examples of suitable circuitryconfigurations that may be employed for optional supplemental imageprocessing circuitry 223 include, but are not limited to, digital logiccontained within an FPGA, ASIC, or standalone integrated circuit (“IC”)or algorithms running on a DSP or other suitable processing element,etc.

In the illustrated embodiment, Frame store circuitry 224 may be anycircuitry suitable for storing higher resolution frames received from asuitable image source, in this case conditioning circuitry 221 (or fromoptional supplemental image processing circuitry 223 when present), andfor providing these frames to Window circuitry 225/Image Scalercircuitry 226 and/or Image Deconstruction circuitry 228 as may beappropriate for a given application or operating mode of same. In thisregard, Frame store circuitry 224 may be configured to store or stagehigh resolution frames and then provide them to Image Deconstructioncircuitry 228 where high resolution image frames are segmented (orseparated into tiles) by circuitry 228. Furthermore, Frame storecircuitry 224 may be configured with bypass circuitry so that highresolution frames received from conditioning circuitry 221 or optionalsupplemental image processing circuitry 223 may be directly provided toWindow circuitry 225/Image Scaler circuitry 226 without frame storage.Alternatively a switch or other suitable selection mechanism (not shown)may be provided with corresponding signal paths that together areconfigured to selectably route high resolution frames to either of Framestore circuitry 224 or directly to Window circuitry 225/Image Scalercircuitry 226 without frame storage. Examples of suitable circuitryconfigurations that may be employed for Frame store circuitry 224include, but are not limited to, SDRAM or SRAM either standalone orcontained within an ASIC or FPGA, etc.

Window circuitry 225 and Image Scaler circuitry 226 may be any circuitryconfigurations suitable for respectively extracting portions of a higherresolution image and scaling frames of a higher resolution imageobtained from Frame store circuitry 224 (or alternatively receiveddirectly from conditioning circuitry 221 or from optional supplementalimage processing circuitry 223 when present) to frames of a lowerresolution image (e.g., NTSC, PAL, SECAM, etc. compatible format), andfor providing these scaled lower resolution frames to Image Muxcircuitry 232. Alternately selected areas may be scaled up for largerdisplay on the monitor 208 via one of the transmission interfaces 206.The advantage of this alternate path is the ability to send higher framerate scaled images at the same time the lower effective frame ratehigher resolution images are being transmitted. Examples of suitablecircuitry configurations that may be employed for Window circuitry 225and Image Scaler circuitry 226 include, but are not limited to, digitallogic contained within an FPGA, ASIC, or standalone IC or algorithmsrunning on a DSP or other suitable processing element, etc.

Image Deconstruction circuitry 228 may be any circuitry suitable forsegmenting (or separating into tiles) the high resolution image framesreceived from Frame store circuitry 224 into two or more multiple highresolution segments, performing appropriate buffering operations toprevent latency issues from arising, and for providing the segmentedhigh resolution frames or tiled high resolution images to Alignment DataInsert circuitry 230. Examples of suitable circuitry configurations thatmay be employed for Image Deconstruction circuitry 228 include, but arenot limited to, digital logic contained within an FPGA, ASIC, orstandalone IC or algorithms running on a DSP or other suitableprocessing element, etc.

Alignment Data Insert circuitry 230 may be any circuitry suitable forinserting tile identification information, horizontal alignmentinformation, and/or vertical alignment information into the unused linesof the segmented frames or tiled images received from ImageDeconstruction circuitry 228. Examples of suitable circuitryconfigurations that may be employed for Alignment Data Insert circuitry230 include, but are not limited to, digital logic contained within anFPGA, ASIC, or standalone IC or algorithms running on a DSP or othersuitable processing element, etc.

Image Mux circuitry 232 may be any circuitry suitable for selecting thescaled lower resolution frames from Image Scaler circuitry 226 or thehigher resolution tile images from Alignment Data Insert circuitry 230for transmission using multiple stream (i.e., two or more stream)format. In this regard, Image Mux circuitry 232 may provide the selectedimage information to conversion circuitry 244. In one embodiment, ImageMux circuitry 232 may be configured to include digital buffering andswitching circuitry and may be characterized as a media router. Examplesof suitable circuitry configurations that may be employed for Image Muxcircuitry 232 include, but are not limited to, digital logic containedwithin an FPGA, ASIC, or standalone IC or algorithms running on a DSP orother suitable processing element, etc. In an alternative embodiment,analog multiplexing may be used. It will be understood that Image Muxcircuitry 232 may be configured to output multiple signal streams, e.g.,to multiple buffers and/or DAC circuits for transmission across one ormore analog interfaces 206.

As illustrated in FIG. 2, multiple instances of conversion circuitry 244may be optionally provided to output separate signal streams (e.g. eachbeing multiple or single streams themselves) to separate respectiveanalog interfaces. It will be understood, however, that multipleinterfaces 206 need not be present however, and that a single analoginterface 206 may be employed. It will further be understood that one ormore digital transmission interfaces (e.g., such as described elsewhereherein) may alternatively be employed and, in one embodiment, may beemployed in combination with one or more analog transmission interfaces.Use of multiple transmission interfaces may be employed to allowincreased frame rate and/or resolution to be transmitted simultaneously.For example, using two analog interfaces 206 allows multiple imagecreation circuitry 202 to transmit a four-tile HDTV image at aneffective frame rate of 7.5 fps over a first analog interface 206 a,while at the same time transmitting a full standard definition image at30 fps over a second analog interface 206 b. In such a case, the HDTVimage and the standard definition image may be displayed simultaneouslyon the same display device, displayed on separate respective displaydevices, and/or stored.

It will further be understood that the disclosed systems and methods maybe advantageously practiced in combination with one or more digitaltransmission interfaces, e.g., to reduce bandwidth requirements. In oneembodiment, for example, the disclosed systems and methods may beadvantageously implemented in combination with a digital transmissioninterface where bandwidth capacity is an issue, e.g., due to thesupported rate of a particular connection or a particular series ofconnections, due to cost associated with bandwidth, due to a need tomultiplex several video signals over one digital link (e.g., due totopology constraints, cost issues, right of way, physical access andavailability, etc).

As further illustrated in FIG. 2, one or more embedded processors 298(e.g., multiprocessor, DSP, or other suitable processor/s) may bepresent in camera 202 for implementing one or more tasks (e.g.,algorithms) described herein and related to one or more individualcircuitry components 221, 223, 270 and 244. As shown in FIG. 2, camera202 may also include a conversion block 244 for digital to analogencoding (e.g., conversion from digital video to standard composite NTSCanalog video) for transmission across interface 206.

Although FIG. 2 illustrates one exemplary embodiment including multiplestream image creation circuitry that is provided within a camera 202 itwill be understood that it is possible that multiple stream imagecreation circuitry may be implemented in any other suitable form orconfiguration, e.g., as circuitry provided within a camera or ascircuitry provided wholly or in-part separate to a camera, as circuitrywith any other suitable configuration of optics and/or image sensor, ascircuitry that is operably coupled for receiving signals from non-cameravideo sources, combinations thereof, etc. Furthermore, it will beunderstood that in other embodiments multiple stream image creationcircuitry (e.g., configured within a camera, configured as separatecircuitry, etc.) may be implemented with other types of image processingcircuitry.

It will also be understood that multi-stream image processing componentsmay be implemented using any alternative circuit configuration suitablefor providing any one or more of the respective capabilities thereof asdescribed herein. In this regard, one or more of the exemplaryillustrated components of multi-stream image processing circuitry 270(e.g., 224, 226, 232, 228 and 230) may be alternatively rearrangedand/or combined in any alternative configuration suitable forimplementing the functionality described herein, either in relation toeach other, and/or in relation to other circuitry (e.g., such asoptional image processing circuitry 223, optional conditioning circuitry221, conversion circuitry 244, etc.). Furthermore, it is possible thatadditional circuitry components may be provided in the path betweenmultiple image creation circuitry and a coupled bandwidth-limitedtransmission interface as may be needed or desired for other purposeswithout departing from the scope of the disclosed systems and methods.In this regard, examples of such circuitry include, but are not limitedto, image storage circuitry, one or more types of image conditioningcircuitry, one or more types of supplemental image processing circuitry,etc.

Still referring to the exemplary embodiment of FIG. 2, video system 200is configured with multiple stream image receiving circuitry in the formof PC-based DVR 204. As illustrated in FIG. 2, PC-based DVR 204 includesframe grabber circuitry 250 for analog to digital conversion of analogsignals (e.g., multi-stream analog signals) transmitted from enhancedresolution digital camera 202 across interface 206. Frame grabbercircuitry 250 is shown coupled to bandwidth limited interface 206 viamultiport interface component 248, although coupling via any othersuitable type of interface component is possible. DVR 204 is also shownprovided with multi-stream image processing circuitry 290 that iscomplementary to multi-stream image processing circuitry 270 of camera202. In this regard, components of multi-stream image processingcircuitry 290 of the exemplary embodiment of FIG. 2 include ImageReconstruction circuitry 252 that may be used to reconstruct tiled imagesegments back into the original high resolution image, Multi StreamCompression circuitry 254 that may be used to compress both the standardand high resolution images to decrease storage requirements, and MultiImage/Stream Storage circuitry 256 that may be used to store the raw orcompressed images, e.g., based on system or system operatorrequirements.

As shown in FIG. 2, for this exemplary embodiment Image Reconstructioncircuitry 252 and Multi Stream Compression circuitry 254 may beimplemented together on processor 260 (e.g., PC-based microprocessorsuch as an Intel Pentium) of PC-based DVR that may be in turn coupled tomultiple image storage 256, which may be any data storage device orcombination of data storage devices suitable for storage of raw orcompressed image information, e.g., such as hard or floppy disk/s,EEPROM, FLASH, DRAM, SRAM, etc. However, it will be understood that theconfiguration of multi-stream image processing circuitry illustrated inFIG. 2 is exemplary only, and that components thereof may bealternatively implemented on more than one processor and/or usingdedicated circuitry (e.g., such as PCI-based standalone compression andimage processing cards), either of which that may be in whole or in partconfigured external to other components of a DVR. Furthermore, it willbe understood that multiple stream image receiving circuitry may beimplemented using non-PC based DVR circuit configurations, or using anyother configurations of one or more circuits (e.g., including non-DVRcircuit configurations) suitable for providing any one or more of therespective capabilities of multiple image processing circuitry asdescribed herein. For example, it will be understood that capabilitiesof a frame grabber component may be combined with multi-stream imageprocessing circuitry, and that a multiple image storage component is notrequired, e.g., multiple image information may be processed and/ordisplayed only in real time, multiple image information may be processedand then provided without storing to other and separate image processingcircuitry for additional image processing, etc.

In the illustrated embodiment of FIG. 2, frame grabber circuitry 224 maybe any circuitry suitable for capturing a frame from an analog videosignal (e.g., multi-stream analog signal) received across interface 206,converting it to a digital image, and providing it to multi-stream imageprocessing circuitry 290. Examples of suitable circuitry configurationsthat may be employed for frame grabber circuitry 224 include, but arenot limited to, industry standard video decoder IC's coupled to asynchronization and memory controller coupled to SDRAM for videostorage, etc.

Image Reconstruction circuitry 252 may be any circuitry suitable forreconstructing tiled image segments back into the original highresolution image originally segmented by Image Deconstruction circuitry228 using alignment data inserted by Alignment Data Insert circuitry 230of enhanced resolution digital camera 202. Examples of suitablecircuitry configurations that may be employed for Image Reconstructioncircuitry 252 include, but are not limited to, PC or DSP basedprocessing with or without dedicated hardware assist, etc.

Multi-Stream Compression circuitry 254 may be any circuitry suitable forcompressing both standard and high resolution image information receivedacross interface 206 and/or processed by image reconstruction circuitry252. In the illustrated embodiment, Multi Stream Compression circuitry254 may provide such compressed image information to Multi StreamStorage circuitry 256, for example, to decrease storage space requiredfor the image information in Multi Stream Storage circuitry 256. MultiStream Storage circuitry 256 may additionally or alternatively providecompressed image information to other system or subsystem components,e.g., for purposes of storage or subsequent transmission to otherviewing stations via LAN (Local Area Network) or WAN (Wide AreaNetwork). Examples of suitable circuitry configurations that may beemployed for Multi Stream Compression circuitry 254 include, but are notlimited to, FPGA or ASIC based logic, dedicated standalone MPEG, JPEG,JPEG2000 integrated circuits, etc.

In the embodiment of FIG. 2, Multi Stream Storage circuitry 256 may beany data storage device suitable for storing lower and higher resolutionimages, e.g., standard and high resolution images. Some examples ofsuitable data storage devices have been previously described herein. Inone exemplary embodiment, lower and/or higher resolution images may beselectably stored in either raw and/or compressed form, e.g., based oncommand or control signals provided by a system operator or anothersystem or another subsystem component. For example, it may be desirableto store images in raw form in those cases where any lossy form ofcompression would adversely affect post processing such as facialrecognition, or target detection, or cases where the image is to besignificantly digitally enlarged before analysis (e.g. retrieval of alicense number off of a car tag that does not represent a significantportion of the overall image, personnel identification either by zoomingin on a face of other aspects of the persons apparel, etc.).Alternatively, images may be stored in compressed form such as when itis desired that the amount of data be decreased in order to reducestorage requirements (e.g., to reduce cost or to increase the amount ofimage time span that may be stored) or to enable a higher frame rateover slower communications links (e.g., modem, ISDN, etc.) than would bepossible with non-compressed video.

In the embodiment of FIG. 2, multiple resolution image display 208 isillustrated as a single display device (e.g., computer monitor or othersuitable digital image display device) configured to simultaneouslydisplay standard resolution image 209 and higher resolution image 211received from DVR 204. However, it will be understood that in thepractice of the disclosed systems and methods that three or moremultiple images of different resolutions and/or frame rates may besimultaneously displayed on a single display device in real time and/orin playback form from recorded data. It will also be understood that twoor more multiple images of different resolutions and/or frame rates maybe displayed on multiple display devices (e.g., standard resolutionimage displayed on a first display device and high resolution imagesimultaneously displayed on a second display device) in real time and/orin playback form from recorded data. Furthermore, it will be understoodthat display of multiple images is not necessary in the practice of thedisclosed systems and methods. For example, one or more images may bedisplayed in real time while other non-displayed image information isfurther processed or recorded for future playback or processing.Alternatively, it is possible that no image is displayed in real time,with all image information being recorded and/or processed by otherseparate image processing circuitry.

In one embodiment of the practice of the disclosed systems and methods,a multiple resolution video system (e.g., such as illustrated anddescribed in relation to FIG. 2) may be implemented in a manner thatallows image information received from one or more image sources (e.g.,image sensor/s, storage device/s containing previously generated andstored image information, etc.) to be selectably processed in a mannerso as to create two or more image streams based thereupon that havedifferent resolutions and/or frame rates. In this regard, frames of eachcreated image stream may correspond to an entire image received from animage source or may correspond to any portion thereof. In this regard,one or more whole or partial images may be segmented into two or moretile segments for transmission across an analog interface. Multipleimages (e.g., higher and lower resolution images) and/or tile segmentsthereof may be simultaneously transmitted across an analog interface bytransmitting each of the multiple images at respective frametransmission rates that together do not exceed the maximum frametransmission rate capacity of the interface.

For example, in one example implementation of the embodiment of FIG. 2,image sensor 222 may provide high resolution image information relatingto a scene viewed by sensor 222 to multi-stream image processingcircuitry 270 of camera 200. Multi-stream image processing circuitry 270of enhanced resolution digital camera 202 may in turn generate astandard resolution image frame of the entire scene from the highresolution image information (e.g., using circuitry related to imagescaling, segmentation, and multiplexing). This standard resolution imageframe may then be transmitted via digital to analog conversion circuitry244 to DVR 204 across interface(s) 206 that has a maximum frame ratetransmission capacity (e.g., 30 frames per second) at a first frame rate(e.g., 15 frames per second) that is less than the maximum frame ratecapacity of interface 206. Simultaneously, higher resolution segmentedimage tiles associated with respective portions of the high resolutionimage scene may also be transmitted in analog form across interface 206by multi-stream image processing circuitry components 270 in camera 202via conversion circuitry 244 to DVR 204 at a second frame rate (e.g., 15frames per second) that is also less than the maximum frame ratecapacity of interface 206, and such that the first and second framerates do not together exceed the maximum frame rate transmissioncapacity (e.g., 30 frames per second) of interface 206.

Once received by DVR 204, frame grabber 250 may provide both thestandard resolution and high resolution image information in digitalform to multi-stream image processing circuitry 290 of DVR 204.Microprocessor-based image processing circuitry 252 may then beconfigured to reassemble and display the scene of the standardresolution image at its transmitted frame rate (e.g., 30 frames persecond) and/or to display the high resolution scene assembled from thesegmented tiles at a frame rate corresponding to the transmission rateof the individual tiles divided by the number of tile segments perimage. For example, a high resolution image segmented into four tilesegments that are transmitted across interface 206 at a rate of 15frames per seconds may be reassembled into a single high resolutionimage that may be processed and/or displayed at a rate of 3.75frames/sec. In an alternative example, a high resolution-only mode maybe implemented that allows for transmission of four high resolutionframes at a total rate of 30 frames per second from multi-stream imageprocessing circuitry components 270 of camera 202 to multi-stream imageprocessing components 290 of DVR 204, for reassembly into a single highresolution image at a rate of 7.5 frames/sec, i.e., four frames that areassembled to make up each overall frame of the full image aretransmitted every 7.5 seconds.

Tile Reassembly to Form Reconstructed Image

In a further exemplary embodiment of the disclosed systems and methods,two or more multiple image tiles may be transmitted separately (e.g.,across standard CCTV composite video interface or other suitablestandard or non-standard interface/s) and may be reassembled into areconstructed image (e.g., high resolution full 1280×720 image). Forexample, in one exemplary embodiment, such a methodology may beimplemented in multi-resolution video system 200 of FIG. 2 to transmitmultiple high resolution image tiles (e.g., each high resolution imagetile being mapped into a respective standard TV resolution video frame)between camera 202 and DVR 204 across analog interface 206 forreassembly into a reconstructed full high resolution image with littleor no visible artifacts. However, it will be understood that such amethodology may be implemented in any other single ormultiple-resolution video system environments having any otherconfiguration of one or more circuit blocks suitable for performing oneor more of the video signal processing tasks described herein.

In those embodiments where transmission of high resolution images (e.g.,1280×720 images) exceed the capacity of the selected analog transmissionmedia interface (e.g., NTSC, PAL, etc.), tiled high resolution imagesmay be transmitted across the analog interface at a frame rate that isequal to the frame rate of the analog interface divided by the number oftiled images transmitted. For example, NTSC transmission media islimited to 720×480 rectangular pixel (“RP”) video frames at 29.97 fps,and PAL transmission media is limited to 720×576 rectangular pixels(“RP”) video frames at 25 fps. Thus, where four high resolution tiledimages are transmitted across the selected analog interface, theresultant effective frame rate of the reassembled high resolution imagemay be one-fourth the actual frame transmission rate (e.g., 30/4 or 7.5fps for NTSC standard, or 25/4 or 6.25 fps for PAL standard).

It will be understood that the capture/archive rate of multiple streamimaging circuitry may not equal the full frame rate of the coupledtransmission interface. For example, referring to the embodiment of FIG.2, PC-based DVR 204 may not capture/archive frames at the full framerate of analog interface 206 (e.g., 30 fps NTSC, 25 fps PAL). Therefore,without compensation, there may be mismatches between the capture rateof DVR 204 and the timing of the received frames containing the tiledhigh resolution images. For example, a DVR capturing at a 2 fps ratewould capture/archive one out of every 15 frames for an NTSCtransmission, and one out of every 12.5 frames for a PAL transmission.In an embodiment where a single high resolution image is transmitted bymultiple image stream circuitry in the form of four tiled images in arepeating 1-2-3-4 pattern, there may therefore be a mismatch between themultiple stream image creation circuitry and the coupled multiple streamimage receiving circuitry. In one exemplary embodiment, such mismatchesmay be compensated for by adapting the frame generation pattern of themultiple stream image processing circuitry (e.g., camera 202) to complywith the capture/archive rate of the coupled multiple stream imagereceiving circuitry (e.g., DVR 204).

The following table illustrates one exemplary manner (i.e., for NTSCstandard) in which compensation may be implemented by repeatingtransmission of each frame containing a given image tile repetitivelyback-to-back manner for a sufficient number of times to allowcapture/archive of at least one frame containing the given image tilebased on the capture/archive rate of the multiple image stream receivingcircuitry, it being understood that similar methodology may be employedfor PAL standard and that other compensation scheme methodologies arealso possible.

Frame Transmission Repetition Rate (Number DVR NTSC of Times A GivenFrame is Resulting Effective Capture Rate Repeatedly TransmittedReassembled Image Rate (fps) Back to Back) (fps) 30 fps 1 7.5 15 2 3.7510 3 2.5

The disclosed systems and methods may be advantageously employed totransmit and reassemble tiles in a manner that takes advantage of extraor left-over information space that exists when a image having a firstresolution is transmitted using a format that has a second resolutionthat is greater (e.g., greater number of pixels and/or greater number ofrows) than the first resolution. For example, referring to the exemplaryvideo system 200 of FIG. 2, an exemplary 1280×720 square pixel HDTVsensor 222 of system 200 may be employed in combination with exemplaryand conventional video encoder/decoder parts that are based onrectangular pixel formats corresponding to CCIR-656 video streams. Forthese encoder/decoder parts the resolution of the NTSC image is about720×480. It will also be understood that 720×480 rectangular TV pixels(“RP”) corresponds to approximately 640×480 square pixels (“SQP”).

For the combination of this given example, a 1280×720 image generated bythe HDTV sensor 222 may be divided by Image Deconstruction circuitry228, for example, into four 640×360 tiles for transmission using theNTSC format. Thus, insertion of each 640×360 high resolution tile into astandard NTSC format image leaves 80 horizontal pixels and an additional120 lines available for alternate (e.g., non-video) purposes, such asdescribed below. It will be understood that this specific combination of640×360 tiles with NTSC format is exemplary only, and that the disclosedsystems and methods may be implemented with any combination of tile sizethat is smaller relative to the transmission frame size (e.g. PALformat, etc.), such that one or more lines and/or horizontal pixels areavailable for alternate purposes as further described herein.

FIGS. 4A-4C illustrate one exemplary embodiment as it may be employed inthe reassembly of four 640×360 tiles 402, 404, 406 and 408 that havebeen inserted for transmission in four respective NTSC frames 422, 424,426 and 428, e.g., by Image Deconstruction circuitry 228 of system 200or other suitable circuitry. As shown in FIG. 4A, each tile 402, 404,406 and 408 may be vertically centered with a vertical line alignmentpattern 410 inserted into one of the available lines preceding theactive image, e.g., by Alignment Data Insert circuitry 230 of system 200or other suitable circuitry configuration. In the illustrated embodimentof FIG. 4A, each vertical alignment pattern is also shown provided withan optional tile identifier pattern 416 therein, e.g., also inserted byAlignment Data Insert circuitry 230 of system 200 or other suitablecircuitry configuration. In such an embodiment, a vertical linealignment pattern 410 with tile identifier pattern 416 may be used touniquely identify each tile for reassembly with other tiles into alarger image 400, e.g., as upper left quadrant tile of the larger image,upper right quadrant tile of the larger image 400, etc. In addition, theplacement of the alignment pattern 410 in the video stream may be usedfor proper vertical alignment and reassembly of tile with adjacenttile/s, e.g., by Image Reconstruction circuitry 252 of video system 200or other suitable circuitry configuration. For example, a verticalalignment pattern 410 may be defined such that the active image of each640×360 tile starts at a fixed number of lines (e.g., 16 lines) from thevideo line that this vertical alignment pattern occupies. In anotherexemplary embodiment, vertical blanking information area, such as thatassociated with closed caption, may alternatively or additionally beused to indicate that the field/frame is a “tile” associated with alarger image, as well as to indicate the relative spatial relationshipof the tile with respect to the original larger image.

It will be understood that the foregoing examples of vertical alignmentpatterns are exemplary only, and that other configurations of verticalalignment information and/or tile identifier information (e.g.,configured as one or more other types of patterns and/or codes) may beemployed that are suitable for vertical alignment and/or identificationof a given tile with respect to other tiles of a given image. Forexample, a tile identifier pattern or other identifier information maybe employed that is separate from a vertical alignment pattern or othervertical alignment information employed for the same tile.

Also shown in the exemplary embodiment of FIG. 4A is how each tile 402,404, 406 and 408 may be configured to support horizontal alignmentinformation, in this example in the form of horizontal alignmentpatterns 412 e.g., inserted by Alignment Data Insert circuitry 230 ofsystem 200 or other suitable circuitry configuration. In the illustratedembodiment, a horizontal alignment pattern 412 may be configured, forexample, to occur temporally to the right of each of the tilesassociated with the left of the original image and to the left of eachof the tiles associated with the right of the original image.

One exemplary horizontal alignment pattern may be as simple as a set offour 3 pixel wide pulses (pure white pixels on a black background) thatmay be used (e.g., by Image Reconstruction circuitry 252 of video system200 or other suitable circuitry configuration) to allow the alignment ofthe segments or tiles horizontally, e.g., via simple correlation. Forexample, as shown in FIG. 4B, a horizontal alignment correlation valuemay be obtained by horizontally shifting lines 412 a and 412 b ofrespective horizontally adjacent tiles relative to each other (asindicated by the arrow in FIG. 4B) and performing a standard correlationfunction 450. In such a case, the peak correlation 452 occurs when thepulses 430 a and 430 b for horizontally adjacent tiles are at maximumalignment with respect to each other. This methodology may be used totake into account the sampling jitter that may occur when the incominganalog image is being re-sampled by a video decoder chip over multipleframe times.

In a further exemplary embodiment, each tile 402, 404, 406 and 408 mayalso be configured to include a small horizontal “overlap” area 414,e.g., consisting of 16 pixels, e.g., by Alignment Data Insert circuitry230 of system 200 or other suitable circuitry configuration. Once thetiles 402, 404, 406 and 408 are properly aligned using horizontalalignment patterns 412 via the horizontal alignment technique describedabove, the respective overlap areas 414 b and 414 a of horizontallyadjacent tiles (e.g., tiles 402 and 404, and tiles 406 and 408) may becombined (e.g., using fading transfer function 440 and summation andlimiting function 442) as indicated in FIG. 4C to form the final largerimage 400 shown in FIG. 4D. This combination may be performed, forexample, by Image Reconstruction circuitry 252 of video system 200 orother suitable circuitry configuration. It will be understood thattechniques other than the simple linear approximation approachillustrated may be employed, e.g., non-linear techniques such as squareroot or logarithmic, or techniques that take into account the imagecontrast around the combined area such as adaptive filtering.

It will be understood that the foregoing examples of horizontalalignment patterns and methodology are exemplary only, and that otherconfigurations of horizontal alignment information (e.g., configured asone or more other types of patterns and/or codes) may be employed thatare suitable for horizontal alignment and/or identification of a giventile with respect to other tiles of a given image. It will also beunderstood that alignment and overlap blending may also occur verticallyto allow for the case where the tiles are first compressed and storedand then later retrieved, decompressed, and reassembled. This allows forany visual artifacts that may occur due to the compression technique atthe image boundaries to be masked. If vertical blending is performedthen the tile identifiers may also be used for alignment purposes.

To maximize resolution and bandwidth availability, it will be understoodthat the tile reassembly techniques described herein may be extended toutilize more of the used pixel area through the proper application ofimage scaling. For example, the 640 image may be horizontally scaled to704 before transmission, which still allows for the 16 pixels requiredfor the pattern and overlap area shown in FIGS. 4B and 4C.

It will be understood that the forgoing example utilizing 1280×780 pixelHDTV resolution is exemplary only, and that other image segmentation andreconstruction configurations may be employed to fit a givenapplication. For example, a 1920×1080 HDTV resolution image may bedivided into six 640×480 tiles and reconstructed into a single image ina manner similar to described above.

Image Signal Processing for Transmission Across Bandwidth LimitedInterface

FIG. 8A illustrates one exemplary embodiment of image data logic flow asmay be implemented, for example, using multi-stream image processingcircuitry components of camera 202 of FIG. 2 or using other suitableconfiguration of multi-stream image processing circuitry. In addition tothe exemplary circuitry embodiment of FIG. 2, it will be understood thatthe image data logic flow illustrated in FIG. 8A may be implemented withany alternative circuitry configuration suitable for accomplishing oneor more of the image data processing tasks described in relationthereto.

As shown in FIG. 8A, a digital image data 833 may be received from anysuitable image source 860 and then travel down one of four exemplarypaths before being selected by the frame multiplexing operations 863 forfinal processing for transmission across a bandwidth-limited interfacein a manner as previously described. This exemplary video flow may beemployed, for example, to support the various selectable display modesof Examples 1-4 described herein. In this embodiment, multiple windowsand scaling functions may be implemented to provide for multiple zoomedor non-zoomed images which may then be combined into a single image (orframe) for analog transmission or sent as sub-images within subsequentframes. These windowing and scaling mechanisms may be implemented bylogic contained within an FPGA or ASIC, or alternatively may beperformed by a DSP or other suitable high speed processing element.

In the illustrated embodiment of FIG. 8A, image source 206 may providedigital image data 833 (e.g., full high resolution image data of1280×720 pixels at 30 frames per sec or fps). Image source 860 may beany source of a suitable image, such as is described elsewhere herein.For example, image source 860 may be an image storage device, may be animage sensor that produces raw image data that may be converted intodesired digital video form (e.g., standard format digital video form) byconditioning circuitry, etc. As illustrated, image source 860 mayprovide digital image data 833 for multi-stream image processing 861,e.g., as may be performed by multi-stream image processing circuitry 270of FIG. 2 or other suitable multi-stream image processing circuitryconfiguration. Digital image data 833 may be in any suitable digitalimage data form and, in one exemplary embodiment, may be digital videodata having an image size of 1280×720 square pixels at 30 frames/second(fps), although digital image data 833 may also be of greater than orlesser resolution and/or greater than or lesser frame rate in otherembodiments. Although digital image data 833 may originate from an imagesource such as image sensor 222 of FIG. 2, it will be understood thatdigital image data may be received for multi-stream image processingoperations 861 from any other suitable image source configuration, suchas described elsewhere herein. Also possible is optional additionalimage processing (e.g., such as performed by optional supplemental imageprocessing circuitry 223 of FIG. 2) that may be performed prior toproviding digital image data for multi-stream image processing 861.

Still referring to the exemplary embodiment of FIG. 8A, multi-streamimage processing 861 may be implemented to provide four possibleprocessing paths for digital image data 833, which for purposes of thisdiscussion will be assumed to represent a high resolution image of adesired scene. Selection of one or more of these paths may beaccomplished, for example, using a switch, data router, or othersuitable selection mechanism 829 provided within circuitry configuredfor performing multi-stream image processing operations 861. Forexample, referring to multi-stream image processing circuitry 270 ofFIG. 2, bypass circuitry may be provided within frame store circuitry224 or as separate switchable data provided from conditioning circuitry221/optional supplemental image processing circuitry 223 to either oneof frame store circuitry 224 or window circuitry 225/image scalercircuitry 226.

In first data path 801, digital image data 833 may be downscaled indownscaling operations 862 (e.g., by image scaling circuitry 226 of FIG.2) to produce downscaled video data 835 of the entire desired scene instandard resolution, e.g., an image scene size of 640×360 square pixelsat 30 fps, for transmission as data 835 across analog interface 206 ofFIG. 2 after frame multiplexing operations 863 that may be performed,for example, by image mux circuitry 232. In such an embodiment, whenreceived and processed by multiple stream image processing circuitry(e.g., PC-based DVR 204), data 835 may be displayed (e.g., on imagedisplay 208 of system 200), for example, as a 640×360 standardresolution image of the entire desired scene at 30 fps.

In second data path 803, frames of the entire desired scene of digitalimage data 833 may be stored or staged in frame store operations 864(e.g., by frame store circuitry 224 of FIG. 2) and then provided in datapath 837 for image deconstruction and insertion of alignment data at 865(e.g. image deconstruction circuitry 228 and alignment data insertcircuitry 230 of FIG. 2). In this regard, an entire frame of the desiredscene or other type of image may be segmented into multiple tiles (e.g.,four quadrants), and appropriate alignment data may be inserted at 864for image reconstruction purposes and the resulting tiled imageinformation provided as tiled image data 839 (e.g., an image tile sizeof 640×360 square pixels at 30 fps) for transmission as data 839 acrossanalog interface 206 of FIG. 2 after frame multiplexing operations 863.When received and processed by multiple stream image receiving circuitry(e.g., PC-based DVR 204 of FIG. 2), data 839 may be displayed (e.g., onimage display 208 of FIG. 2), for example, as a reconstructed 1280×720high resolution image of the entire desired scene at 30 fps.

In third data path 805, a selected portion of the desired scene or otherimage may be extracted from frames of digital image data 833 duringwindowing operations 866 (e.g., performed by window circuitry 225 ofFIG. 2) to form a windowed partial image of the desired scene or othertype of image, e.g., an image window size of 640×480 square pixels at 30fps, for transmission as data 841 across analog interface 206 afterframe multiplexing operations 863. When received and processed bymultiple stream image receiving circuitry (e.g., PC-based DVR 204), data841 may be displayed (e.g., on image display 208 of FIG. 2), forexample, as a 640×480 image of the entire desired scene at 30 fps.

In fourth data path 807, a selected portion of the desired scene orother image may be extracted from frames of digital image data 833during windowing operations 867 (e.g., by window circuitry 225 of FIG.2) to form a windowed partial image of the desired scene or other image,e.g., an image window size of 320×240 square pixels at 30 fps, which isrepresented by windowed partial image data 810 in FIG. 8A. The windowedpartial image of data 810 may then be upscaled during upscalingoperations 868 (e.g., by image scaler circuitry 226 of FIG. 2) to form azoomed partial image of the desired scene or other type of image, e.g.,a 2× zoomed image window size of 640×480 square pixels at 30 fps, fortransmission as data 845 across analog interface 206 after framemultiplexing operations 863. It will be understood that in oneembodiment, the amount of magnification of a zoomed image window may becontrolled by the size of the portion of the desired scene that isselected for extraction in windowing operations 867 prior to upscalingoperations 868 (e.g., when upscaling to a fixed image size in upscalingoperations 868). However, it is also possible to control the amount ofupscaling (e.g., by varying the size of the upscaled image producedduring upscaling operations 868). When received and processed bymultiple stream image receiving circuitry (e.g., PC-based DVR 204 ofFIG. 2), data 845 may be displayed (e.g., on image display 208 of FIG.2), for example, as a 1280×960 zoomed image of the entire desired sceneat 30 fps. It is also understood that less multiple smaller segments ofthe larger image may be transmitted to 204 of FIG. 2 via 850 andindividually scaled up to larger images at 204.

During frame multiplexing operations 863 (e.g., performed by Image Muxcircuitry 232), data is selected for transmission (e.g., data 835, 839,841 and/or 845) and may be output as selected image data 850 for furtherappropriate buffering and analog encoding for transmission across one ormore analog interfaces (e.g., analog interface 206 of FIG. 2). In thisregard identity of the multiple streams transmitted are determined byselection of data 835, 839, 841 and/or 845 during frame multiplexingoperations 863.

It will be understood that the illustrated embodiment of FIG. 8A isexemplary only, and that any other combination of different number andtypes of image streams and stream operations may be implemented. In thisregard, the number of data paths may be greater or lesser than four, andany given data path may represent any one or more window or scalingoperations suitable for achieving desired image characteristics.Furthermore, although not illustrated in FIG. 8A, it will be understoodthat the frame rate of any given image data path may vary as may bedesired in order to meet the needs of a given application (e.g., toallow transmission of the desired amount of image information within thebandwidth of an analog interface.

Tile Identifier Pattern Insertion and Detection

As previously described, image video signal 839 of FIG. 8A may containtiled image information (e.g., two or more tiles) for transmissionacross an analog interface/s to suitable circuitry for reconstruction orreassembly of the multiple tiles into a single image. Such tiledinformation may be created, for example, by image deconstructioncircuitry 228 and alignment data insert circuitry 230 of FIG. 2 or byany other suitable configuration of one or more circuits using, forexample, the signal processing flow described in relation to signal 839of FIG. 8A or any other suitable signal processing methodology.

FIG. 9 illustrates one exemplary embodiment of a representative 640×480de-interlaced active frame 900 having a vertical line alignment pattern410 that has been inserted into one of the available lines preceding theactive image 910. As previously described, vertical alignment pattern410 may also be provided with an optional tile identifier pattern 416.In this regard, FIGS. 11A-11D illustrate examples of tile identifierpatterns 416 such as they may be employed in the practice of oneexemplary embodiment of the disclosed systems and methods. FIG. 9 alsoshows right tile overlap area 912 and left tile overlap area 914 as theymay be together present when frame 900 is a center tile, e.g., of a sixtile image. It will be understood that only one of right tile overlaparea 912 or left tile overlap area 914 need be present when frame 900 iseither a right tile or left tile, e.g., of a four tile image. As shownin FIG. 9, vertical alignment pattern 410 with optional tile identifierpattern 416 may also be present below active tile area 910 in order toserve the purpose of vertical blending of the upper and lower tiles. Asfurther shown, non-active tile areas 913 may be black.

With regard to horizontal overlap area format definition, active tilearea 910 of frame 900 of FIG. 9 has been segmented from an original1280×720 image. In order to split the 1280×720 image into four separate640×360 tiles and reconstruct the original image without presenting anoticeable seam or other visible artifact an overlap area 912 or 914 maybe defined that is 8 pixels wide, i.e., defined on the right border of aleft tile or defined on the left border of a right tile. This overlaparea 912 or 914 may contain 4 pixels that are repeated for both the leftand right tiles, and 4 pixels that are used as a horizontal alignmentmechanism 412. Alternatively, both overlap areas 912 and 914 may bedefined on respective left and right borders in the case of a centertile.

Still referring to FIG. 9, it will be understood that additional and/oralternative types of tile identification information may be provided tofurther identify individual tile segments of an original image and/or tosynchronize transmission and receipt of a given identified tile acrossan interface, e.g., to synchronize or coordinate camera circuitry 202with frame grabber circuitry 250 of video system 200 of FIG. 2, such asunder conditions where camera circuitry 202 and frame grabber circuitry250 are turned on at different times. In one exemplary embodiment,additional lines of frame 900 may be employed to contain additional tileidentifier information (not shown in FIG. 9) to distinguish sequentialtransmitted frames containing respective sequential tiled images fromeach other, such as four sequentially transmitted frames of a four-tileimage such as shown in FIG. 4A. For example, line 28 may be transmittedalternately as full white and full black to indicate frame transmissionsequence, e.g., line 28 may be full white to indicate that a given frame900 is transmitted as frame number one or as frame number three, andline 28 may be full black to indicate that a given frame 900 istransmitted as frame number two or as frame number four. Upon receipt ofeach frame 900 (e.g., by frame grabber 250), line 28 may be examined todetermine whether the given frame is a first or third transmitted frame,or is a second or fourth transmitted frame. It will be understood thatadditional identification information may be provided to furtheridentify or specify the sequential identity of a given frame 900, e.g.,line 27 may be alternately full white and full black in combination withline 28 to specify whether a given frame 900 is transmitted as framezero (e.g., lines 27 and 28 both white), one (e.g., line 27 white, line28 black), two (e.g., line 27 black, line 28 white) or three (lines 27and 28 both black).

FIG. 13 illustrates one exemplary image reconstruction embodiment thatmay be implemented to reconstruct or reassemble an image from multipletiles transmitted across an analog interface according to the disclosedsystems and methods. The methodology of FIG. 13 may be implemented, forexample, using multiple stream image receiving circuitry in the form ofDVR 204, or using any other circuitry configuration suitable for imagereconstruction/reassembly. As illustrated in FIG. 13, an incoming analogvideo signal (e.g., analog video signal received across analog interface206 from camera 202) containing image tiles of a desired video scene maybe received and converted from analog to digital signal form in framegrab step 890 (e.g., by frame grabber circuitry 250 or other suitablecircuitry). In optional selection step 899, image tiles from frame grabstep 890 may be selected for further processing, or alternatively,stored image tiles from an image storage source (e.g., image storage 256or other suitable image storage device) may be selected for furtherprocessing.

Still referring to FIG. 13, tile identifier information of the multipleimage tiles associated with a given multi-tile image selected in step899 may be detected in step 891, and gathered together in step 892(e.g., by Image Reconstruction circuitry 252 of video system 200). Inthe event that one or more tiles are received out of sequential order,they may be re-queued for processing as shown in step 898.

Next, the gathered multiple tiles of step 892 may be horizontallyaligned for assembly into the full multi-tile image in step 893 usinghorizontal alignment information included in the individual multipletiles. In optional step 894, the gathered multiple tiles may bevertically aligned for assembly into the full multi-tile image based onoptional vertical alignment information that may be included in thetiles. In step 895, tiles received from step 893 (or from optional step894 when present) may be horizontally blended together, followed byoptional vertical blending of the tiles when appropriate in optionalstep 896. The blended (reconstructed) image may then be displayed (e.g.,on image display 208 or other suitable display device/s) in step 897. Itwill be understood that the particular steps and sequence of stepsillustrated and described in relation to FIG. 13 are exemplary only, andthat other sequences, and/or additional or fewer steps may be employed.For example, horizontal alignment and blending steps 893 and 894 may besequentially interchanged with vertical alignment and blending steps 895and 896, etc.

With regard to the image reconstruction embodiment illustrated anddescribed in relation to FIG. 13, FIG. 10 illustrates placement of atile alignment pattern 410 with tile identifier pattern 416 when tworespective tiles 1020 and 1022 are vertically assembled to form avertical frame 1024, for example, by image reconstruction circuitry 252of FIG. 2 or by any other suitable configuration of one or morecircuits. As shown, a tile alignment pattern 410 and tile identifierpattern 416 may be present above and below the active tile areas 1010 ofeach of tiles 1020 and 1022, with a vertical blanking area 1030 beingprovided above the active tile areas 1010 of each of tiles 1020 and1022. A horizontal blanking area 1032 may be present to the left of theactive tile areas 1010 of each of tiles 1020 and 1022. These blankingareas may be optionally employed to contain other information such ascamera ID, time stamp, etc.

Referring again now to FIGS. 9 and 11A-11D, following is an exemplarysequence of steps that may be employed to verify that frame 900 of FIG.9 represents a high resolution tile instead of normal NTSC/PAL video,and to determine which tile segment (e.g., upper left, upper right,lower left, lower right) is contained therein:

-   -   Step 1: Verify lines 0 thru 28 and lines 31 thru 59 of frame 900        are full black;    -   Step 2: Verify lines 29 and 30 of frame 900 contain one of the        patterns 1002, 1004, 1006 or 1008 of FIG. 11, e.g., via        correlation;    -   Step 3: While performing step 2, store which pattern (i.e.,        1002, 1004, 1006 or 1008) was detected on lines 29 and 30; and    -   Step 4: Analyze the pattern detected in step 3 to determine the        tile placement.

With regard to Step 4 of the above sequence, exemplary patterns 1002,1004, 1006 or 1008 of respective FIGS. 11A-11D each contain informationthat identifies both vertical and horizontal positioning of the tilesegment of frame 900 within an image assembled from four tile segments.In this regard, a left pattern portion 1102 and a right pattern portion1104 may be provided for each pattern. As illustrated, either the leftpattern portion 1102 or the right pattern portion 1104 of each tile maybe provided as an alternating white and black pattern to indicatehorizontal positioning of the given tile, i.e., whether the given tileis a left or right tile segment. For example, patterns 1002 and 1006each have right pattern portions 1104 that repeatedly alternate betweenfull white and full black to indicate that the tile is a right pattern.In the same manner, patterns 1004 and 1008 each have left patternportions 1102 that repeatedly alternate between full white and fullblack to indicate that the tile is a left tile segment. Verticalpositioning of a given tile may be indicated by the remaining patternportion that is not used to indicate the horizontal positioning of thetile. For example, patterns 1002 and 1004 each have remaining patternportions, 1102 and 1104 respectively, that have a non-broken full whitepattern to indicate that the given tile is an upper tile. Patterns 1006and 1008 each have remaining pattern portions, 1102 and 1104respectively, that alternate twice between full white and full black toindicate that the given tile is a lower tile.

It will be understood that the illustrated patterns and patternconfigurations of FIGS. 9-11 are exemplary only and that any otheralignment and/or identifier pattern or other form of alignment and/oridentifier information may be employed that is suitable for aligningadjacent tiles and/or identifying placement of a given tile within agiven reconstructed image. For example, other forms of suitableidentifier information include, but are not limited to identifier code,time stamp, camera identification, camera position in the case of a PTZcamera, etc. Other forms of suitable alignment information include, butare not limited to, square wave patterns, single pulse, ramp, smoothcurve, etc. Furthermore, it will be understood that alignment and/oridentifier information may be adaptive, meaning that the pattern can becommanded to change based on the type of compression method that is usedor how stable the frame grabber interface is.

Since the standard DVR decoder produces 640×480 images, the resultantreconstructed image horizontally will be less than 1280 pixels. Forexample, in this described four-tile embodiment, a total of 8 pixels areused for alignment purposes and a total of 8 pixels are used to create ablended 4 pixel output. Therefore, the resultant image horizontalresolution will be 1268 pixels (i.e., 1280 total pixels—8 pixelsalignment—4 pixels lost due to overlap). In terms of horizontal pixelsnumbered from 0 to 639 for each left and right tile, the horizontalstructure of the reconstructed image of this embodiment may becorrelated to pixels 0 to 1279 of the original image and characterizedas follows:

Left Tile (upper or lower): Pixel 0-631 = pixels 4 thru 635 of theoriginal image Pixel 632-635 = pixels 636 thru 639 of the original 1280image Pixel 636 = alignment pattern pixel 0 = full black Pixel 637 =alignment pattern pixel 1 = full white Pixel 638 = alignment patternpixel 2 = full black Pixel 639 = alignment pattern pixel 3 = full whiteRight Tile (upper or lower): Pixel 0 = alignment pattern pixel 0 = fullblack Pixel 1 = alignment pattern pixel 1 = full white Pixel 2 =alignment pattern pixel 2 = full black Pixel 3 = alignment pattern pixel3 = full white Pixel 4-7 = pixels 636 thru 639 of the original 1280image Pixel 8-639 = pixels 640 thru 1271 of the original image Note:Original image pixels 0-3 and 1272-1279 may be discarded.

Although implementation of a horizontal overlap area is described inthis example, it will be understood that an overlap area may be providedat one or more edges of a given video frame adjacent one or morerespective borders of a tile segment contained in said given videoframe, regardless of orientation (e.g., horizontal overlap area,vertical overlap area, etc.).

It will be understood that the preceding text describes one exemplaryembodiment of the disclosed systems and methods that may be employed inthe separation of an image into multiple (e.g., four) individual orseparate tiles, and in the reconstruction of the image from the multipleseparate tiles. It will be understood that this embodiment is exemplaryonly, and that other tile alignment configurations, tile segment sizes,original image sizes and/or number of tile segments may be employed tosegment an original image into two or more multiple tile segments and toreconstruct at least a portion of the original image by assembling themultiple tile segments together.

Transmission Across Bandwidth Limited Analog Interface for Display onAnalog Display Device

FIG. 3A illustrates another embodiment of the disclosed systems andmethods employing multiple resolution image creation circuitry that maybe coupled to create and provide multiple image information in multipleresolutions via analog signals for display on one or more analog displaydevices. In this regard, FIG. 3A illustrates one exemplary embodiment ofa video system 2000 having multiple resolution image capability that maybe implemented in the practice of the disclosed systems and methods, forexample, to create and provide multiple-resolution image transmissioncapability over a conventional analog transmission interface 2060.

As shown in FIG. 3A, video system 2000 may include multiple resolutionimage creation circuitry in the form of a digital camera 2020 thatincludes multiple resolution image processing components (2702, 2704,2706, 2708, 2710, 2712, 2714 and 2716), the features of which may beimplemented using any suitable hardware and/or software configuration(e.g., digital signal processor (“DSP”), application specific integratedcircuit, field programmable gate array, combinations thereof, etc.).Also shown in FIG. 3A is DVR 2040 coupled between analog interface 2060and analog display device 2080, in this case a standard analog displaymonitor 2080. Note that the DVR is not necessary for proper display onthe monitor 2080. As shown in FIG. 3A, video system 200 may beimplemented in one embodiment to provide a multiple resolution imagedisplay on monitor 2080 (e.g., simultaneous display of an entirestandard resolution scene 2082 and three selected zoomed areas 2084,2086 and 2088 of the scene in the area beneath the entire scene 2082).

Although FIG. 3A illustrates a multiple resolution video system 2000capable of providing a particular multiple resolution image capability(e.g., in this exemplary embodiment simultaneous display of entirenon-zoomed scene at standard resolution with three zoomed resolutionareas displayed below the entire scene), it will be understood that thedisclosed systems and methods may be alternatively implemented toprovide a video system capable of providing any combination of zoomedand/or non-zoomed resolution images. For example, one or more zoomedareas may be simultaneously displayed with a non-zoomed area, or two ormore zoomed areas may be simultaneously displayed alone (i.e., withoutsimultaneous display of non-zoomed area). Furthermore, the particularsize of any given zoomed or non-zoomed area relative to the size ofother images that are simultaneously displayed may be varied as sodesired, e.g., three zoomed areas may be displayed in a space occupyinggreater than half of the overall display space simultaneously with anon-zoomed image that occupies less than half of the overall displayarea. In addition, positioning of any given zoomed or non-zoomed arearelative to positioning of other images that are simultaneouslydisplayed in the same display space may also be varied as so desired,e.g., three zoomed areas may be simultaneously displayed above anon-zoomed area. Thus, it will be understood that the particularcombination of zoomed and/or non-zoomed images simultaneously displayedon a common display, as well as the relative positioning and sizethereof, may be varied (e.g., varied in real time or on a pre-determinedbasis) as needed or desired to meet the requirements of a givenapplication.

In FIG. 3A, enhanced resolution digital camera 2020 is shown coupled toDVR 2040 and analog display device 2080 by analog interface 2060 (e.g.,similar to the analog interface 106 of FIG. 1). In this regard, analoginterface 206 may be any signal transmission format suitable fortransmitting analog video signals. In one exemplary embodiment, analoginterface 206 may be a typical coaxial cable-based analog videotransmission format (e.g., standard composite video transmissioninterface). In this embodiment, analog interface is not necessarilybandwidth limited.

It will be understood that a standard composite video analogtransmission interface is only one example of a suitable analoginterface. Other examples of analog interfaces which may beadvantageously employed to transmit a desired image signal usingmulti-resolution methodology include, but are not limited to, fiberoptic, RF, etc. It will also be understood that additional processing orconversion may be employed as long as the transmission containsmulti-resolution information in accordance with the systems and methodsdisclosed herein. Also note that in all cases simple viewing of theresultant image can be performed without the use of the DVR 2040.

As illustrated for the exemplary embodiment of FIG. 3A, digital camera2020 includes optics 220 and an image source in the form of video orimage sensor 222, in this exemplary embodiment a high resolution videosensor having a standard HDTV resolution of 1280×720 (as opposed to thestandard 720×480 or 720×576 resolution of the video sensor of prior artdigital camera 102 of FIG. 1). Although a HDTV high resolution sensor isillustrated and described in relation to the exemplary embodiment ofFIG. 3A, it will be understood with benefit of this disclosure that adigital camera may be provided with any other type and/or resolution ofhigh resolution or standard resolution image sensor or combination ofsuch sensors that are suitable for generating image information havingcharacteristics described further herein. Examples of suitable imagesensors include, but are not limited to, image sensors havingresolutions of 1920×1080, 1280×1024, 2048×2048, etc. Suitable types ofimage sensors include, but are not limited to, CCD, CMOS, ultraviolet,near infrared, infrared, etc.

Although video system 2000 of the exemplary embodiment of FIG. 2 isconfigured having a camera with an image source in the form of an imagesensor, it will be understood that a multiple resolution video systemmay be configured in the form of a camera or in a non-cameraconfiguration with any one or more types of image sources that aresuitable for generating image information in any picture format, e.g.,including image sensors, storage devices capable of providing previouslygenerated and stored image information, high frame rate images generatedfrom RADAR or laser mapping, etc.). Furthermore, an image source may bea digital image source, or may be an analog image source producing ananalog signal that is converted to digital signal via digital to analog(“DAC”) conversion. In one embodiment, the disclosed systems and methodsmay be implemented to simultaneously receive, process and interleaveimage signals from two more image sources, e.g., image signalsoriginating from wide-angle view and narrow angle view image sensors,image signals from IR and visible spectrum image sources, image signalsfrom two image storage devices, etc.

As shown in FIG. 3A, camera 2020 may be configured with conditioningcircuitry 221 that is capable of converting information from sensor 222into video information having a desired digital video form. In thisregard, conditioning circuitry 221 may be any circuitry suitable forconverting raw sensor information into desired digital video form (e.g.,standard format digital video form) including, but not limited tocircuitry capable of converting CCD or CMOS sensor information (e.g.,the bayer pattern red-green-blue (“RGB”), or alternatelycyan-magenta-yellow (“CMY”)) into planar RGB or CMY. Other possibletasks which may be performed by conditioning circuitry 221 include, butare not limited to, the control of a video sensor to produce the propersignal levels (exposure control), providing the correct synchronizationof sensor control signals, and other conditioning aspects such asaperture correction (i.e. high frequency boost), white balance, colorcorrection, demosaicing, dynamic range processing, formatting, etc.Color space conversion such as RGB or CMY to 4:4:4, 4:2:2 or 4:2:0format YCrCb may also be performed by circuitry 221. In one exemplaryembodiment, such conditioning processing may be performed via embeddeddigital logic or via algorithms running on a DSP or any other suitableprocessing element. It will be understood that type/s and combination/sof types of conditioning may vary and may be performed by circuitry 221as desirable to fit the characteristics of a given application. It isalso possible that no conditioning circuitry may be present, forexample, in the case of a digital video signal received from a videosource in the desired signal form (e.g., received in the form of astandard format digital video signal).

Not shown in FIG. 3A are optional image processing circuitry 223 andembedded processor/s 298, either or both of which may be present incamera 2020 to perform similar tasks as described in relation to camera202 of FIG. 2.

As illustrated for the exemplary embodiment of FIG. 3A, multipleresolution image creation circuitry in the form of camera 2020 may beconfigured with multi-resolution image processing circuitry 2700 thatincludes Window circuitry components 2702, 2706 and 2710 coupled torespective image scaler circuitry components 2704, 2708 and 2712. Eachof Window circuitry components 2702, 2706 and 2710 is configured toextract a selected portion of the higher resolution frame to form awindowed partial image, and each of image scaler circuitry components2704, 2708 and 2712 is configured to scale the higher resolution imageto a desired image size for inclusion into a NTSC, PAL, SECAM, etc.compatible format. In this regard, it will be understood that the zoomedresolution of a given image may be controlled by varying the extractedwindow size relative to the degree of image scaling between any givenpair of coupled window and image scaler circuitry components (e.g., toachieve a zoomed, non-zoomed, or fractionally zoomed image).

It will be understood that multiple instances of window circuitrycomponents (e.g., window circuitry components 2702, 2706 and 2710) andrespectively coupled multiple instances of image scaler circuitrycomponents (e.g., image scaler components 2704, 2708 and 2712) may beprovided in order to implement separate image data paths that may bemerged for transmission in a manner as will be described further herein,for example, in relation to FIG. 8B. Furthermore, as shown in FIG. 3A,within a given data path multiple instances of window circuitrycomponents (e.g., multiple window circuitry components 2710) andrespectively coupled multiple instances of image scaler circuitrycomponents (e.g., multiple image scaler components 2712) may be providedin order to produce multiple windowed and scaled image areas within agiven common data path that may be merged for transmission with imagesof other data paths in a manner as will be described further herein, forexample, in relation to FIG. 8B. In the latter case, sequence controlregister circuitry 2714 (e.g., containing window definition registersand corresponding scaling definition registers) may be provided todirect sequential windowing and scaling operations for each of themultiple zoomed image areas created within a common data path, as willbe described further herein in relation to FIG. 8B.

It will be understood that the number of instances of window circuitrycomponents and respectively coupled image scaler components may bevaried as needed or desired to fit the requirements of a givenapplication, e.g., to provide more than three separate data paths, toprovide less than three separate data paths, to implement more than onedata paths that each have the capability to produce multiple zoomedimage areas in the same common data path, etc.

Window circuitry components 2702, 2706 and 2710 and respective Imagescaler components 2704, 2708 and 2712 may be implemented using anycircuitry configurations suitable for respectively extracting portionsof an image (e.g., higher resolution image) and for scaling the framesof the extracted image to frames having a desired resolution (e.g., toachieve the desired zoom magnitude) and having a resolution that iscompatible with the video transmission interface (e.g., NTSC, PAL,SECAM, etc. compatible format), and for providing these scaledresolution frames to Frame Buffer circuitry 2716. Examples of suitablecircuitry configurations that may be employed for Window circuitrycomponents 2702, 2706 and 2710 and respective Image scaler components2704, 2708 and 2712 include, but are not limited to, digital logiccontained within an FPGA, ASIC, or standalone IC or algorithms runningon a DSP or other suitable processing element, etc.

Frame Buffer circuitry 2716 may be any circuitry suitable for receivingand buffering frames from Image scaler components 2704, 2708 and 2712,for selectively merging two or more of these received frames into acomposite video frame and for routing these frames, for example, in amanner described further herein in relation to FIG. 8B. In this regard,Frame Buffer circuitry 2716 may provide the selected image informationto conversion circuitry 244 for digital to analog encoding (e.g.,conversion from digital video to standard composite NTSC/PAL, etc.analog video) for transmission across interface 2060. In one embodiment,Frame Buffer circuitry 2716 may be configured to include digitalbuffering and switching circuitry and may be characterized as a mediarouter. Examples of suitable circuitry configurations that may beemployed for Frame Buffer circuitry 2716 include, but are not limitedto, digital logic contained within an FPGA, ASIC, or standalone IC oralgorithms running on a DSP or other suitable processing element, etc.In an alternative embodiment, analog multiplexing may be used. Althoughnot illustrated, it will be understood that Frame Buffer circuitry 2716may be configured to output multiple signal streams, e.g., to multiplebuffers, media bus interfaces, and/or DAC circuits for transmissionacross one or more analog interfaces 2060.

Although FIG. 3A illustrates one exemplary embodiment including multiplestream image creation circuitry that is provided within a camera 2020 itwill be understood that it is possible that multiple resolution imagecreation circuitry may be implemented in any other suitable form orconfiguration, e.g., as circuitry provided within a camera or ascircuitry provided wholly or in-part separate to a camera, as circuitrywith any other suitable configuration of optics and/or image sensor, ascircuitry that is operably coupled for receiving signals from non-cameravideo sources, combinations thereof, etc. Furthermore, it will beunderstood that in other embodiments multiple resolution image creationcircuitry (e.g., configured within a camera, configured as separatecircuitry, etc.) may be implemented with other types of image processingcircuitry.

It will also be understood that multi-resolution image processingcomponents may be implemented using any alternative circuitconfiguration suitable for providing any one or more of the respectivecapabilities thereof as described herein. In this regard, one or more ofthe exemplary illustrated components of multi-resolution imageprocessing circuitry 2700 (e.g., 2702, 2704, 2706, 2708, 2710, 2712,2714, 2716) may be alternatively rearranged and/or combined in anyalternative configuration suitable for implementing the functionalitydescribed herein, either in relation to each other, and/or in relationto other circuitry (e.g., such as optional image processing circuitry223 (not shown), optional conditioning circuitry 221, conversioncircuitry 244, etc.). Furthermore, it is possible that additionalcircuitry components may be provided in the path between multipleresolution creation circuitry and a coupled analog transmissioninterface as may be needed or desired for other purposes withoutdeparting from the scope of the disclosed systems and methods. In thisregard, examples of such circuitry include, but are not limited to,image storage circuitry, one or more types of image conditioningcircuitry, one or more types of supplemental image processing circuitry,etc.

Still referring to the exemplary embodiment of FIG. 3A, video system2000 is shown configured with optional DVR 2040 coupled between analoginterface 2060 and analog display device 2080. In this regard DVR 2040may be optionally present, for example, to record incoming analog imagedata and/or for optionally implementing electronic pan-tilt-zoom(“EPTZ”) capability as described further herein in relation to FIG. 8B.It will be understood, however, that the presence of DVR 2040 isoptional and that in other embodiments an analog display device, such asstandard analog monitor 2088, may be directly coupled to an analoginterface (e.g. coaxial cable-based analog interface 2060) for directlyreceiving analog video images from multiple resolution image creationcircuitry such as camera 2020. Alternatively, it is possible that analogvideo images received across an analog interface from multipleresolution image creation circuitry may be recorded by analog and/ordigital recording device for later display, i.e., without real timedisplay of the image information as it is received. It will also beunderstood that analog video recorder devices, additional analogdisplays and other image processing and/or display devices may beoperably coupled to process or display image information received acrossthe analog interface.

FIG. 8B illustrates one exemplary embodiment of image data logic flow asmay be implemented, for example, using multi-resolution image processingcircuitry components of camera 2020 of FIG. 3A. In addition to theexemplary circuitry embodiment of FIG. 3A, it will be understood thatthe image data flow illustrated in FIG. 8B may be implemented with anyalternative circuitry configuration suitable for accomplishing one ormore of the image data processing tasks described in relation thereto.

As shown in FIG. 8B, a digital image data 833 may be received from anysuitable image source 860 and then travel down one of three exemplarypaths 8010, 8050, 8070 before being selected during frame buffer/mergeoperations 8630 for final processing for transmission across aninterface in a manner as previously described. This exemplary video flowmay be employed, for example, to support the display mode of FIG. 5described herein. In this embodiment, multiple windows and scalingfunctions may be implemented to provide for one or more zoomed imageswhich may be transmitted across an analog interface for display on ananalog display device. These windowing and scaling mechanisms may beimplemented by logic contained within an FPGA or ASIC, or alternativelymay be performed by a DSP or other suitable high speed processingelement.

In the illustrated embodiment of FIG. 8B, image source 860 may providedigital image data 833 (e.g., full high resolution image data of1280×720 pixels at 30 fps). Image source 860 may be any source of asuitable image, such as is described elsewhere herein. For example,image source 860 may be an image storage device, may be an image sensorthat produces raw image data that may be converted into desired digitalvideo form (e.g., standard format digital video form) by conditioningcircuitry, etc. As illustrated, image source 860 may provide digitalimage data 833 for multi-resolution image processing operations 1461,e.g., as may be performed by multi-resolution image processing circuitry2700 of FIG. 3A or other suitable multi-resolution image processingcircuitry configuration. Digital image data 833 may be in any suitabledigital image data form and, in one exemplary embodiment, may be digitalvideo data having an image size of 1280×720 square pixels at 30frames/second (fps), although digital image data 833 may also be ofgreater than or lesser resolution and/or greater than or lesser framerate in other embodiments. Although digital image data 833 may originatefrom an image source such as image sensor 222 of FIG. 3A, it will beunderstood that digital image data may be received for multi-resolutionimage processing operations 1461 from any other suitable image sourceconfiguration, such as described elsewhere herein. Also possible isoptional additional image processing (e.g., such as performed byoptional supplemental image processing circuitry 223 of FIG. 2A) thatmay be performed prior to providing digital image data formulti-resolution image processing 1461.

Still referring to the exemplary embodiment of FIG. 8B, multi-resolutionimage processing operation 1461 may be implemented to provide threepossible processing paths for digital image data 833, which for purposesof this discussion will be assumed to represent a high resolution imageof a desired scene. Selection of one or more of these paths may beaccomplished, for example, using a 1 to N switch or mux or othersuitable selection mechanism 1429 provided within circuitry configuredto perform multi-resolution image processing operations 1461. It will beunderstood that the embodiment of FIG. 8B is exemplary only and that itis possible that more than three, or less than three, data processingpaths may be provided in other embodiments.

In first data path 8010, digital image data 833 may be downscaledwithout previous windowing in downscaling operations 8620 (e.g., byimage scaler circuitry 2704 of FIG. 3A) to produce downscaled video data8350 of the entire desired scene in standard resolution (e.g., an imagescene size of 640×360 square pixels at 30 fps) and which may then beprovided for merging with other image data as part of merged image data8500 (e.g., by frame buffer circuitry 2716 of FIG. 3A) and thenformatted and converted to suitable analog form (e.g., by conversioncircuitry 244 of FIG. 3A) for transmission across the analog standardvideo interface 206 of FIG. 3A. In such an embodiment, when theanalog-converted image information from downscaled video data 8350 isreceived by analog display device (e.g., standard analog monitor 2080 ofFIG. 3A), the downscaled video data 8350 may be displayed, for example,as a 640×360 standard resolution image of the entire desired scene at 30fps (e.g., as upper 640×360 image 2082 on 640×480 analog image display2080 of FIG. 3A). In such a case, a space of 640×120 is left under image2082 in which other merged image data may be displayed as will befurther described.

Still referring to first data path 8010 of FIG. 8B, no windowingoperations are performed so that a downscaled image of the entire scenemay be obtained. Regarding the downscaling operations 8620, downscalingmay be performed to take scale the high-resolution input stream 833 andscale it symmetrically into a 640×360 output stream @ 30 frames/sec. Itwill be understood that the use of symmetrical scaling is optional, butmay be employed to retain the visual aspect ratio of the input imagedata 833 while at the same time matching the analog video interfacestandards-based output buffer requirement. In one exemplary embodiment,a scaling ratio of 0.5 may be applied to both the X and Y axes of theoriginal image, thus rendering a ¼ scaled image that is smaller butidentical in proportion to the input images.

As previously mentioned, the 640×360 frames of image data 8350 may thenbe forwarded to Frame Buffer/merger operations 8630 where they may bemerged into standard-sized frame buffers of 640×480. In one exemplaryembodiment, when so configured, scaled output frame buffers from imagedata 8450 may be merged in Frame Buffer/merge operations 8630 with the640×360 frame buffers of image data 8350 into a standard 640×480 outputframe buffer such that a composite analog video stream may be composedfrom both of these video data logic paths for transmission across astandard analog video interface. Such an exemplary embodiment may beused to implement the Zoom-Under-Picture (“ZUP”) display capabilitiesillustrated by zoomed images 2084, 2086 and 2088 under downscaled image2082 displayed on standard analog monitor 2080 of FIG. 3A (and furtherillustrated with regard to Example 5 herein). It will be understood thatthe 640×480 frame buffers described herein are standard TV resolutionframe buffers but in a PC format that uses square pixels. In thisregard, it will be understood that Frame Buffer/merge operations 8630may also merge incoming video stream input data from two or more of dataflow logic paths 8350, 8410 and 8450 into standard 720×480 frame bufferssuitable for direct output on standard composite video (TV) media. FrameBuffer/merge operations 8630 (e.g., frame buffer circuitry 2716 of FIG.3A) may also be configured to adapt and/or merge incoming video datastreams and to route the resultant video output streams to one, or more,video media interfaces and conversion circuitry.

In second data path 8050 of FIG. 8B, a selected portion of the desiredscene or other image may be extracted from frames of digital image data833 during windowing operations 8660 (e.g., performed by windowcircuitry 2706 of FIG. 3A) to form a windowed partial image of thedesired scene or other type of image. The windowed partial image of data8100 may then be optionally scaled during optional scaling operations8680 (e.g., by image scaler circuitry 2708 of FIG. 3A) to form a zoomedor non-zoomed partial image of the desired scene or other type of imagefor optional merging with other image data as part of merged data 8500(e.g., by frame buffer circuitry 2716 of FIG. 3A) and then formatted andconverted to suitable media form (e.g., by conversion circuitry 244 ofFIG. 3A) for transmission across media interface 206 of FIG. 3A.

With regard to second data path 8050, a window may be extracted from theincoming image data (e.g., 1280×720 at 30 frames/sec) for output as anindependent video stream. In one embodiment, data path 8050 may beimplemented to allow a user to select a ‘window’ of the full resolutionimage and view this region-of-interest as a separate video stream. Thisalso allows for optional scaling for magnification (i.e. ‘zooming’) ofthe selected window of the full resolution image and/or for adaptivescaling to fit the resultant images into a frame buffer resolutionacceptable for the frame buffer processing logic (e.g., 640×360 or640×480) of Frame Buffer/merge operations 8630. In one exemplaryembodiment, two or more multiple image data output streams 8500 a to8500 n may be provided, allowing multiple images or image areas to beprovided by circuitry 1461 for transmission to multiple display devices(e.g., for simultaneous transmission of different image streams acrossmultiple respective transmission interfaces to enable simultaneousdisplay on multiple respective analog display devices).

In one exemplary embodiment, an area of 640×480 pixels may be extractedfrom the upper left quadrant of the full 1280×720 image and then sentout as buffers (e.g., as an independent image data stream 8500 n) on adesignated video media processor via the Frame Buffer processor of FrameBuffer/merge operations 8630. In this exemplary embodiment, no scalingis performed in scaling operations 8680, and instead the windowed imagedata 8100 may be allowed to pass through scaling operations 8680unscaled and in its extracted windowed form. Since such images arepassed through at their original resolution in such an embodiment (andnot scaled by ¼ as described above in relation to one exemplaryembodiment of data path 8010), to meet display resolution requirements,this accomplishes the equivalent of a 4× digital zoom. In this regard,the 4× zoom factor occurs since 4 times the pixels are passed through tothe video media from the same area of the original full resolution imageversus the scaling output performed in data path 8010 described above.

Still referring to second data path 8050 of FIG. 8B, another exemplaryembodiment may be implemented in which a designated 320×240 area of thefull resolution image is extracted in windowing operations 8660 and thenpassed to scaling operations 8680. In this case, since the 320×240resolution is ¼ of the necessary pixels required for display on a TV, orequivalent device, scaling operations 8680 may be employed to scale theimage symmetrically into 640×480 output frame buffers that are readilydisplayable on standard media (this results in the equivalent of an 8×digital zoom factor). In such a case, the original 320×240 image is 4×the number of pixel data normally used to represent the same area onstandard display media (since it wasn't equivalently downscaled), and byscaling the 320×240 images to 640×480, another 4× of magnification isprovided by scaling each pixel into 4 pixels worth of data. To furtherillustrate image manipulation flexibility that may be realized in thepractice of this embodiment of the disclosed systems and methods, seconddata path 8050 may be alternatively implemented without scalingoperation 8680 in combination with one of the following two exemplarychoices (it being understood that the following are exemplary only andthat a wide variety of other image manipulation variations may beimplemented)

-   -   1) No scaling of the 320×240 windowed output data 8100 and        placement of the under-sized image buffers into 640×480 output        buffers readily routable by the Frame Buffer logic of Frame        Buffer/merge operations 8630; or . . .    -   2) Output the undersized 320×240 buffers of windowed output data        8100 to Frame Buffer/merge operations 8630 where Frame Buffer        logic may merge the incoming undersized video frame buffers with        other video frame buffers (in one exemplary embodiment these may        also be undersized) to create a resultant video stream data        comprised of images from several input video stream data paths.        Using this methodology, for example, a Picture-In-Picture        scenario may be implemented where a small video buffer from data        path 8050 is overlayed on a section of a full screen video        buffer from another data path, e.g., data path 8010. In another        example, the output video stream buffers from data path 8050 may        be merged with the scaled output of data path 8070 (to be        described below) to create several video ‘windows’ in composite        video image stream.

In third data path 8070 of FIG. 8B, multiple selected window (e.g.,regions of interest) of the desired scene or other image may beextracted from frames of digital image data 833 during multiplewindowing operations 8670 (e.g., by multiple instances of windowcircuitry 2710 of FIG. 3A) to form multiple windowed partial images ofthe desired scene(s) or other image(s) in image data 8102. Each of theseextracted multiple windowed images of image data 8102 may then beoptionally scaled during multiple scaling operations 8682 (e.g., byrespective multiple instances of image scaler circuitry 2712 of FIG. 3A)to form respective zoomed or non-zoomed partial images of the desiredscene (or other type of image) that is contained in image data 8450.Image data 8450 may then be merged with other image data in framebuffer/merge operations 8630 to form part of merged image data 8500(e.g., by frame buffer circuitry 2716 of FIG. 3A) and then formatted andconverted to suitable media form (e.g., by conversion circuitry 244 ofFIG. 3A) for transmission across analog video or other suitable mediainterface 206 of FIG. 3A.

With regard to third data path 8070, multiple windowing operations 8670and multiple scaling operations 8682 may be performed in a cyclicmanner. In this regard, windowing operations 8670 may be performed usingrespective multiple instances of window definition register operations8071 (e.g., 8071 a to 8071 c) and respective scaling definition registeroperations 8072 (e.g., 8072 a to 8072 c) that may be present, forexample, if multiple settings are contained in these windowing andscaling registers the sequence control register circuitry 2714 of FIG.3A will cycle through these settings to create individually scaledversions of part(s) of the original image. Using this methodology,windowing operations 8670 (e.g. performed by multiple instances ofwindow circuitry 2710 of FIG. 3A) may be configured to useconfiguration, or control, data setup that is contained in respectiveWindow Definition register sets to determine where, and how much, pixeldata to extract from the incoming full resolution frame buffers. In oneexemplary embodiment, Window Definition Registers sets may be used byall Windowing logic units to identify the location of a ‘window’ and itssize. The register notation may be in any suitable form (e.g., format),but in one embodiment a register notation format may be employed thathas an ‘X axis offset’ register, a ‘Y axis offset’ register’, a ‘X axissize/length’ register, and a ‘Y axis size/length register; these fourregisters comprising the Window Definition Registers and defining where,and how large, an image window is. Although three window definitionregister operations 8071 are illustrated in FIG. 8B, it will beunderstood that there may be any number of Windows Definition registersused for defining various ‘windows’ (i.e., areas to be extracted).

Scaling operations 8682 may employ a set of Scaling Definition registerset operations 8072 that each correspond to a respective windowdefinition register set and that define, per each respective input‘window’ video stream of image data 8102, what scaling ratio to employfor generating the resultant output video frame buffers in image data8450. In one exemplary embodiment, Scaling Definition Registers may becommon to all Scaling logic units and may define the scaling ratio forboth the X and Y axes. Similar to multiple windowing operations 8071,multiple Scaling operations 8682 in data path 8070 may be implementedwith multiple scaling definition registers, in contrast to a singlescaling operation that may employ a single scaling definition registers.

As illustrated in FIG. 8B, Window Definition register operations (8071 ato 8071 c) and respective Scaling Definition register operations (8072 ato 8072 c) are logically linked together. Therefore, for each ‘window’defined to be extracted from image data 833, there is a WindowDefinition register set 8071 and a directly corresponding ScalingDefinition register set 8072. Together these corresponding register setsmay be used to identify a specific video data stream for a particularvideo ‘window’ to be extracted. It will be understood that three windowdefinition register sets and the corresponding three scaling definitionregister sets are exemplary only, and that less than three or more thanthree windows may be defined and scaled in other embodiments using acorresponding number of respective window and scaling definitionregisters. It will also be understood that cases may exist wherewindowing is performed with no scaling, etc.

In one exemplary embodiment of data path 8070, all of the defined video‘window’ data streams may be frame multiplexed into a single videooutput data stream. This may be done, for example, by also correlatingthe Window Definition register and Scaling Definition register activitywith their respective processing units via a sequential register controllogic unit in Sequential Register Control operations 8074 (e.g., insequence control register circuitry 2714 of FIG. 3A). In this exemplaryembodiment, a Sequential Register Control operation 8074 may be used todetermine when, and how many, video frames are processed for eachdesignated video ‘window’ stream. In this fashion, it may drive ‘n’number of frames per each video ‘window’ in a continuous cyclic fashion.For example, one may assume that a user has configured three videowindows of 212×120. Since the output video rate of the attached videomedia is 30 frames/second, the Windowing and Scaling operations 8670 and8672 may be implemented to render a total of 30 frames/second. Sincethere are three video ‘windows’ defined by the user for this case, thenthese windows may be processed at an effective rate of 10 frames/secondeach to derive a resultant video data stream 8102 of 30 frames/second.Thus the Sequential Control register operation 8074 may be used to drivewhich Windows Definition register sets and corresponding ScalingDefinition register sets are active at each given interval therebydetermining the multiplexing rate, sequence and overall timing.Optionally, the Sequential Register Control operations 8074 may also beimplemented using an optional signaling or control path to the FrameBuffer processing logic of frame buffer/merge operations 8630 to drive acyclic form of frame buffer merging or placement.

In one exemplary embodiment for creation of multiple zoomed image areasin data path 8070, Sequential Control Register logic may be implementedin sequential control register operation 8074 to evenly subdivide theavailable frame rate (e.g., 30 frames/second video frame rate) among theactive ‘windows’ for purposes of windowing operations 8670 and scalingoperations 8682. Therefore one active zoomed may be implemented with 30frames/second (e.g., when a user selects to implement multiple windowoperation 8670 and multiple scaling operation 8682 to only create onezoomed image area in data path 8070), two active zoomed image areas maybe implemented to each get 15 frames/second, three active zoomed imageareas may be implemented to each get 10 frames/second, and so on. Also,the output video frame buffers may be sent to the Frame Buffer processorfor merging, ‘underneath or below’, the 640×360 frame buffers from Path1 into a composite 640×480 frame buffer output stream.

It will be understood that many other image processing options areavailable using the set of logical operations that comprise data path8070 illustrated in the exemplary embodiment of FIG. 8B. For example, itis possible that Sequential Register Control operation 8074 may beimplemented to drive different video ‘windows’ at longer intervals(e.g., up to several seconds) in a contiguous manner so as to emulate‘camera tours’. In this regard, camera tours are typically performed bytaking video streams from several separate cameras and ‘cycling’ throughthe views on a monitor at a fixed rate of ‘n’ seconds each. In anotherexample, it is possible to create many smaller video ‘windows’ inwindowing and scaling operations 8670 and 8682 and to output these asdata stream 8450 to the Frame Buffer processor or other circuitryemployed in frame buffer/merge operation 8630 along with proper bufferplacement logic such that a video stream which is a composite ofmultiple video ‘windows’ may be generated. For example, six video‘window’ definitions that are sufficiently small (e.g., in the 160×120pixel range) will fit into a 640×480 frame buffer in a ‘2 rows of 3windows each’ configuration. Using a standard ‘round-robin’ approach,the Sequential Register Control logic of sequential register controloperation 8074 may be implemented to update each ‘window’ at aneffective rate of 5 frames/second, though the actual output rate foreach video media may always be at 30 frames/second. The precedingexamples of this paragraph are exemplary only, with it being understoodthat a variety of other image processing methodologies may beimplemented as needed or desired to fit a given application.

Still referring to FIG. 8B, frame buffer/merge operation 8630 may beimplemented to receive input image data streams (data streams 8350, 8410and 8450), that each consist of frame buffers. Frame buffer/mergeoperation 8630 may then be implemented to perform buffer adaptationand/or merging, where needed or specified, and then to route theresultant image data stream to on or more video media interfaces. Inthis regard, in one exemplary embodiment buffer adaptation and/ormerging may performed under the following three exemplary conditions:

1) The frame size of an incoming image data stream may be incompatiblewith a particular video media interface to which it is specified to berouted. For example, data path 8010 may be generating a 640×360 framebuffer stream, but the backend video media requires a 640×480 videostream. In this case, Frame buffer/merge operation 8630 may beimplemented to either: a) Generate the frame data on-the-fly (i.e. bysending the 640×360 frame buffer out and then generating the final640×120 frame data for transmission), to satisfy the outgoing mediarequirements, or b) Place the incoming 640×360 buffers into 640×480frame buffers before transfer to the video media interface.

2) The incoming video frame buffers of a given data path aresignificantly smaller than required by the target video media interface(e.g., such as case where 320×240 input frame buffers are designated tobe transmitted on a video media interface that requires a 640×480 orgreater output frame buffer size). In such a case, the framebuffer/merge operation 8630 may be implemented to provide severalconfiguration options for encapsulating the 320×240 images in 640×480frame buffers. Example of possible encapsulation options include, butare not limited to, center versus top versus bottom justification,border generation, black space generation, etc.;

3) Merging may be performed by capturing incoming frame buffers from twoor more image data streams and placing them into a unified buffer, usingspecified placement information (that may be configured), such that theincoming streams are merged into one outgoing data stream. For example,640×360 input frame buffers of data stream 8350 may be placed at the topof 640×480 frame buffers, and 212×120 input frame buffers of data stream8450 may be placed sequentially along the bottom 640×120 area of thesame 640×480 frame buffers. The buffers may then be adapted, if needed(see above), to the outgoing video media interface and routed to thatmedia device. It will be understood that any number of merging methodsmay be used to created a variety of composite ‘views’ in outgoing videostreams.

As mentioned above, in addition to buffer adaptation and merging, framebuffer/merge operation 8630 may be implemented in one exemplaryembodiment to route the resultant image data streams to one or morevideo media interfaces (e.g., video media devices) based onconfiguration parameters. Such configuration parameters may be fixed,user set, or dynamic. In any case, each output image data stream may beconsidered to represent a logical, or virtual, camera output. Thisrouting capability advantageously may be implemented to enable numerousviewing options. For example, multiple data streams may be routed overone video media interface using a time multiplexed method. With multiplevideo media interfaces, any one data streams (e.g., data stream 8350,8410, 8450) may be configured, either statically or dynamically, to berouted to one or more of the video media interfaces. Furthermore, asdescribed below, each transmission interface may be supported as avirtual camera and additionally support the ability to provide aPan-Tilt-Zoom (“PTZ”) functionality for each image data stream in anindependent manner.

Although not illustrated in FIG. 8B, frame buffer/merge operation 8630may be configured to route the final output image data streams to one ormore video media interfaces. It will be understood that the predominantstandards for TV-related media are analog based, and the primarystandards are NTSC (720×480 @ 30 fps) and PAL (720×576 @ 25 fps).Additionally, these standards assume interlaced video input. Given thesestandards, standard square pixel-based images may be converted torectangular pixel based images as used by NTSC and PAL. This may beimplemented by scaling the images from 640×480 (or whatever the case maybe) to NTSC or PAL format, and then clocking the data to the varioustransmission video media interfaces in either an interlaced orprogressive manner.

It will be understood that the illustrated embodiment of FIG. 8B isexemplary only, and that any other combination of different number andtypes of image streams and image stream operations may be implemented.In this regard, the number of data paths may be greater or lesser thanthree, and any given data path may represent any one or more window orscaling operations suitable for achieving desired image characteristics.Furthermore, although not illustrated in FIG. 8B, it will be understoodthat the frame rate of any given image data path may vary as may bedesired in order to meet the needs of a given application (e.g., toallow transmission of the desired amount of image information within thebandwidth of an analog interface.

In the foregoing discussions, image processing is described that may beimplemented to allow multiple image streams (e.g., video streams) to begenerated at potentially varying resolutions. Also described arescenarios where a given image stream may be a composite of two or moreindividual image data streams. In one embodiment of the disclosedsystems and methods, it is possible to take any of the sub-highresolution ‘windows’ within a given image stream and to control them ina dynamic (i.e., real-time) manner such that one or more of the‘windows’ behaves as an individual PTZ camera. This PTZ methodology maybe implemented in a digital manner to achieve an Electronic PTZ(“EPTZ”)-like or Digital PTZ (“DPTZ”)-like functionality.

Referring to the embodiments of FIGS. 3 and 8B, any given image ‘window’that is smaller than the full resolution image may be moved aroundwithin the spatial dimensions of the full resolution image. For example,a 320×240 video ‘window’ may be moved around significantly within a fullresolution 1280×720 image. By employing the multiple Windowing andScaling processing operations of the disclosed systems and methods asdescribed herein in relation to FIGS. 3 and 8B, ‘windows’ may bedynamically moved around by updating the register values of theWindowing Definition Register operations 8071 in a real-time fashion.The register values of the Scaling Definition Register operations 8072may also be updated in real-time, thus providing digital zoom, ormagnification, of the images in each selected ‘window’. This methodologymay be implemented in any manner suitable for updating the appropriaterespective window and/or scaling register values to achieve the desiredeffect. One exemplary embodiment is described below, although othermethodologies are possible.

Conventional mechanical PTZ cameras are typically controlled via cameracontrol protocols (e.g., Pelco-P, Pelco-D, AutoDome Control Codeprotocol, SpeedDome RS-422/RS-485 protocol, etc.) that are transferredvia serial communications links such as RS-485, RS-422, or RS-232. Thesecamera control protocols are well documented and they enable themanagement of multiple cameras per serial link using serial messagesthat contain unique camera IDs. Camera IDs are also called ‘addresses’,‘device IDs, or ‘camera numbers’, and they uniquely identify thespecific camera on a serial link as the designated target of a cameracontrol message. Each of these protocols enables a set of basic commandssuch as ‘Pan-Right’, ‘Pan-Left’, ‘Tilt-Up’, ‘Tilt-Down’, ‘Zoom-In’(greater magnification), ‘Zoom Out’ (lesser magnification), and so on.

In the practice of the disclosed systems and methods, a serial link maybe attached to multiple resolution image creation circuitry (e.g.,camera 2020 of FIG. 3A) for supporting camera control commands such asdescribed in the preceding paragraph. Additionally, multiple resolutionimage creation circuitry may be configured to emulate multiple camerason a serial link by mapping camera IDs/addresses to specific ‘windows’in its output image data streams (e.g., data streams 8350, 8410 and 8450of FIG. 8B), which may optionally be further mapped to two or morerespective separate multiple image data output streams 8500 (e.g., to8500 a, 8500 b, and 8500 c, respectively) for display on two or moreseparate analog display monitors. FIG. 3B illustrates the implementationof one such exemplary embodiment having a serial link 2310 (e.g.,RS-485, RS-422, RS-232, etc.) that couples a PTZ controller withjoystick 2320 to an optional processor 2330 provided in multipleresolution image creation circuitry 2700 of camera 2020 via serial port2340. As illustrated, processor 2330 is coupled to each window circuitrycomponent (2702, 2706, 2710), and to each image scaler circuitrycomponent (2704, 2708, 2712) in a manner so as to enable selectivecontrol of these components in order to provide the PTZ capabilitiesdescribed further herein.

When optional multiple image data output streams 8500 are provided it ispossible, for example, to display multiple zoomed areas on one analogdisplay device, and then selectably display and control a full screenzoomed image (e.g. of one of the multiple zoomed areas displayed on theother monitor) on another analog display device using camera controlcommands mapped to specific windows. However, any other combination oftwo or more images on two or more display devices is also possible whenmultiple image data output streams 8500 are provided.

In one exemplary embodiment, multiple resolution image creationcircuitry such as camera 2020 of FIG. 3A or 3B may be configured tosupport camera IDs/addresses, e.g., such as camera addresses 3 and 4(for illustration purposes). For example, camera 2020 may map cameraID/address 3 to a 320×240 viewing window in an image data streamprovided by second data path 8050 of FIG. 8B and it may map cameraID/address 4 to one of multiple 212×120 zoomed image areas such asdefined by Windowing Definition Register set operation 8071 and ScalingDefinition Register set operations 8072 in third data path 8070 of FIG.8B. Pan Right/Left/Up/Down commands may be then be provided for eithercamera ID 3 or 4, and the corresponding Window Definition Register setsare modified in Windowing Definition Register operations 8071 to emulatecamera panning within the full resolution image of data stream 833. Forexample, FIG. 14 illustrates how x and coordinates 1500 within a windowdefinition register for a particular windowing operation may be variedin one exemplary embodiment to pan and/or tilt a 320×240 window area1520 within a 1280×720 full resolution image 1530, it being understoodthat these particular window and overall image resolution values areexemplary only.

Using the above methodology, if a Pan command causes a ‘window’ to hitthe viewing boundary of the full resolution image, the ‘window’ may beconfigured in one embodiment to stop moving (i.e., to ignore Pancommands that are operating in the errant direction), or move into‘black space’. When Zoom commands are received for a ‘virtual’ camera,the corresponding Scaling Definition Register set may be modified toeither increase the scaling factors/ratios (i.e., to Zoom-Out) ordecrease the scaling factors/ratios (i.e., to Zoom-In). In this manner,multiple virtual cameras may be emulated in one embodiment by mappingcamera IDs/addresses in standard PTZ camera control protocols to EPTZfunctionality provided by the Windowing and/or Scaling operations (e.g.,8670 and/or 8682 of FIG. 8B) on a per ‘window’ basis, and such that anychosen ‘window’ is spatially smaller than the source full resolutionimage (e.g., full high resolution image of image data path 833 of FIG.8B).

It will be understood that the forgoing EPTZ/DPTZ methodology may beimplemented with regard to any given one or more image data path/s ofmultiple resolution image creation circuitry, regardless of the numberand/or type of data paths implemented therein.

EXAMPLES

The following exemplary and hypothetical examples are illustrative andshould not be construed as limiting the scope of the invention or claimsthereof.

Example 1 Selectable Extraction of Higher Rate High Resolution ImageArea Out of a Lower Rate High Resolution Image for Digital DisplayDevice

FIGS. 5A and 5B illustrate one exemplary embodiment of the disclosedsystems and methods in which a reconstructed high resolution image(e.g., 1280×720) of an entire scene 506 of FIG. 5A may be displayed at areduced frame rate (e.g., at 3.75 frames/second) simultaneously with aselected or selectable portion 508 of the scene (e.g., a 640×480 imagearea) that may be displayed as a high resolution image 510 of FIG. 5B ata higher frame rate (e.g., at 15 frames/sec), e.g., displayed togetheron image display 208 of multi-resolution image video system 200 or onother suitable digital display device/s. In one embodiment, the highresolution image 510 of the selected portion 508 of the scene 506 may beextracted at the camera end prior to transmission, e.g., by imagedeconstruction circuitry 228 and image scaling circuitry 226 of videosystem 200, or other suitable circuitry.

In one embodiment, a RS485 interface or other suitable control interfacemay be provided to allow an operator to select the window area fordisplay of the smaller image at the higher frame rate, e.g., at imagedisplay 208 or at other suitable interface location. In the case shownin FIGS. 5A and 5B, the selected area is 640×480 and represents the fullsensor high resolution image 510 for the selected specific area 508 atany given time. In a further exemplary embodiment, an operator may beprovided with a control interface that allows the operator to slideportion 508 as a selectable window area around the area of full image506 in order to vary the identity of the selected area or portion 508 ofthe scene 506 for display as image 510, e.g., to achieve a 15 frames/secelectronic pan-tilt-zoom (EPTZ) function display of selectable image510. This windowing function may be provided, for example, in imagedeconstruction circuitry 228 of system 200. In one embodiment, embeddedprocessor 298 (e.g., multiprocessor, DSP or other suitable processingcircuitry) may intercept the serial (i.e., RS-485, RS-422, RS-232, etc.)commands that may originate from a control interface (e.g. joystick,keyboard, touch screen) and then direct the image deconstructioncircuitry 228 to selectively sample the overall sensor area. The imagescaler circuitry 226 may also be used to either expand or contract(i.e., scale) the selected area to produce the desired size outputimage. It will be understood that multiple instances of both 226 and 228may be employed to allow for multiple areas at different scalingfactors. In such an embodiment, a control interface may be configured toselect which pair of image scaler circuitry 226 and image deconstructioncircuitry 228 is actively being controlled. This may be accomplished,for example, via interpretation of the serial commands by processingcircuitry 298 of system 200.

Using the above-described technique, one or more selected or selectablehigher frame rate areas may be displayed in any suitable manner, forexample, displayed simultaneously with the entire lower frame rate sceneon a separate monitor or other display device, or simultaneouslydisplayed with the lower frame area on the same monitor or displaydevice (e.g., in a window separate from the entire lower frame ratescene, or superimposed over a corresponding portion of the lower framerate scene in the same window).

In another exemplary embodiment, when a standard resolution CCTV monitoris employed as part of a system (e.g., as a spot monitor), a scaledimage transmitted at a first rate (e.g., at 15 frames/sec) may betemporarily up-scaled to a higher frame rate (e.g., 30 frames/sec) viaframe replication. Furthermore, for display of a HDTV image (i.e.,having 16:9 aspect ratio) on the standard monitor (e.g., having 4:3aspect ratio), a choice may be made to display the image with blankedvertical lines (cinema mode) or by cropping the edges of the images tofit the 4:3 format.

The advantage of this exemplary embodiment is that it may be used toallow an operator to maintain an overall situational awareness whilesimultaneously being able to watch high motion rate events.

Example 2 Selectable Extraction of Zoomed High Resolution Image Area Outof a Larger High Resolution Image for Digital Display Device

FIGS. 5C and 5D illustrate another exemplary embodiment of the disclosedsystems and methods in which a high resolution image (e.g., 1280×720) ofan entire scene 502 of FIG. 5C may be displayed (e.g., at 3.75frames/sec) simultaneously with a selected or selectable zoomed portion504 of the scene that may be displayed as a higher frame rate image 550of FIG. 5D (e.g., at 15 frames/sec or other desirable frame rate), e.g.,displayed together on image display 208 of multi-resolution image videosystem 200 or on other suitable digital display device/s. For example,in the exemplary embodiment of FIG. 5B the windowed area 504 representsa selected or selectable 320×240 area (at the sensor native resolution)that is expanded to 640×480 before being transmitted from the camera,e.g., by image scaling circuitry of video system 200. As in theembodiment of Example 1, window 504 may be optionally movable in aselectable manner by an operator in real time, e.g., to scan the scene502 during an alarm event.

Using the above-described technique, one or more selectable zoomed areasmay be displayed in any suitable manner, for example, displayedsimultaneously with the entire unzoomed scene on a separate monitor orother display device, or simultaneously displayed with the entire sceneon the same monitor or display device (e.g., in a window separate fromthe entire scene). Selectable zoomed image/s may also be displayed in asuperimposed manner over a corresponding portion of the unzoomed scenein the same window and, if desired, appropriate image scaling may beemployed so that the entire scene is viewable in either of the zoomed orunzoomed portions of the total displayed image.

The advantage of this exemplary embodiment is that it may be used toallow an operator to gain more usable detail out of a specific area(e.g., during an alarm event) while simultaneously maintaining visualawareness of a much larger area.

Example 3 Selectable Extraction of Zoomed High Resolution Image Area Outof a Stored Larger High Resolution Image for Digital Display Device

FIGS. 6A-6C illustrate the significant advantages that may be achievedwith stored video content obtained using the higher resolution imagingof the disclosed systems and methods as compared to existing standardresolution imaging. In this regard, FIG. 6A represents an storedunzoomed image taken by a video surveillance camera of an overall scene600 and, for purposes of this example, may be considered to be astandard resolution image, or alternatively a high resolution (e.g.,HDTV) image according to the disclosed systems and methods. Scene 600includes a portion 602 of the scene 600 that corresponds to an event ofinterest (e.g., person of interest, item of interest, activity ofinterest, etc.). FIG. 6B illustrates a 4× digital zoom of portion 602 ofscene 600 that results when starting from a stored unzoomed standardresolution image of scene 600. In contrast, FIG. 6C illustrates theincreased scene detail that is obtained from a 4× digital zoom ofportion 602 of scene 600 that results when starting from an unzoomedHDTV resolution image of scene 600, such as may be provided as describedelsewhere herein across a standard analog interface using the disclosedsystems and methods. In this regard, higher resolution informationcaptured using the disclosed systems and methods may be used to providesubstantially increased detail of a given portion of a video image andto provide greater information as to what was occurring at the timearound an event of interest, even when the video camera is not zoomedinto that given portion of the video image.

Example 4 Simultaneous Monitoring of Zoomed and Unzoomed Images onDigital Display Device

FIG. 7 illustrates a graphic display 700 (e.g., computer monitor), andshows how the overall area around an aircraft may be monitored in afirst window display 702 (e.g., image display 208 of multi-resolutionimage video system 200 or other suitable digital display device/s) tomaintain overall situation awareness, while one or more zoomed images ofspecific portions of the overall video image may be simultaneouslymonitored to observe one or more specific area/s of activity orinterest. Specifically, FIG. 7 shows how multiple zoomed images (secondand third window displays 704 and 706 of the graphic display 700) may besimultaneously monitored with overall area display 702. It will beunderstood that one or more specific portions of an overall video imagemay also or alternatively be viewed simultaneously and/or sequentiallyon one or more graphic displays (e.g., multiple computer monitors orstandard television monitors) as may be desirable or needed in a givenapplication. The resultant display may be formed from post-transmissionprocessing (e.g. in a PC) or pre-transmission processing (e.g. in thecamera) depending on the which system and method is used.

Example 5 Simultaneous Monitoring of Zoomed and Unzoomed Images onAnalog Display Device

FIG. 12 illustrates simultaneous display of a primary image 1202 of anentire or overall scene and three zoomed image areas 1210, 1212 and 1214(taken from respective locations 1204, 1206 and 1208 of primary image1202 of the entire scene) on an analog display device 1200, such asconventional analog display device 2080 of FIG. 3A or 3B or othersuitable analog display device. In the illustrated embodiment, displayof the entire scene in primary image 1202 may be used to maintainoverall situation awareness, while zoomed image areas 1210, 1212 and1214 may be simultaneously monitored to observe more specific area/s ofactivity or interest.

As illustrated, zoomed image areas 1210, 1212 and 1214 do not overlaythe primary image 1202 of the entire scene, but are instead displayedoutside the perimeter of the primary image (i.e., without occluding theviewing space of the primary image). In one exemplary embodiment,primary image 1202 may be displayed so that it occupies the upper 75% ofa standard TV resolution video frame (e.g., displayed as upper 640×360out of total 640×480) displayed on analog display device 1200. In thisembodiment, primary image 1202 represents the entire image source fieldof view (“FoV”), e.g., of image sensor 222 or other suitable imagesource. Because primary image 1202 has a 16:9 aspect ratio, it does notcompletely fill the full vertical image space of a standardTV-resolution video frame (i.e., having a 4:3 aspect ratio) that isdisplayed on analog display device 1200. As such, one or more otherimages may be displayed in the remaining 640×120 space that remains atthe bottom of the TV-resolution video frame. In the illustratedembodiment of FIG. 12, three zoomed image areas 1210, 1212 and 1214 areeach 212×120 SQP images that are displayed side by side beneath primaryimage 1202 in a zoom-zone under picture (“ZUP”) configuration.

In the illustrated embodiment of FIG. 12, each of zoomed image areas1210, 1212 and 1214 may be, for example, fixed size zoomed areas ofselected sections of an original high resolution image prior to beingscaled to 212×120 (e.g., fixed size zoomed areas of an original 1280×720sensor image from image sensor 222 of video system 2000 or othersuitable image source). In one embodiment, zoomed image areas 1210, 1212and 1214 may be selectively zoomed areas (e.g., selected by systemoperator or suitable image control mechanism) of primary image 1202,having locations 1204, 1206 and 1208, respectively, within the primaryimage 1202 as shown. It will be understood that zoomed area/s mayalternatively be of fixed location within primary image 1202 ifselectability is not desired or needed. In any event, whether fixed orselected, zoomed image areas 1210, 1212 and 1214 may be extracted,zoomed, and placed (e.g., by multiple resolution image creationcircuitry 2700 of FIG. 3A or 3B) within a standard TV-resolution videoframe, along with primary image 1202, at the lower section of the videoframe as shown for display on standard analog display device (e.g.,display device 2080 of FIG. 3A or 3B). Further, in one exemplaryembodiment, the number of displayed zoomed image areas, dimensions ofdisplayed zoomed image areas, and/or position of zoomed image areaswithin a given display relative to a primary image may be selectable,either beforehand and/or in real time (e.g., allowing selection of asingle zoomed image area to be centered below primary image 1202, ratherthan display of all three zoomed image areas 1201, 1212, and 1214).

The amount of zoom magnification magnitude of zoomed image areas 1210,1212 and 1214 may be fixed, and/or in one embodiment may be selectivelyvaried (e.g., in real time). For example, in one exemplary embodiment,each of zoomed image areas 1210, 1212 and 1214 may have a zoom factorthat may be selectively varied from about 2× to about 20× magnification,a zoom magnification of about 4× being a default value, althoughmagnification greater than about 20× is also possible. Magnificationless than 1× (i.e., fractional magnification) is also possible, e.g.,entire 1280×720 may be downscaled to a 212×720 image area for display asone of zoomed image areas 1210, 1212 or 1214.

In one exemplary embodiment, all active zoomed image areas (e.g., zoomedimage areas 1210, 1212 and 1214 of FIG. 12) may share a video scalingunit within hardware scaling circuitry. In this regard, hardware scalingcircuitry may be configured to operate at the appropriate frame rate forthe transmission standard employed, e.g., 30 fps for NTSC, 25 fps forPAL, etc. Thus, the frame rate for each given zoomed image area may beequal to the total frame rate (e.g., NTSC, PAL, etc.) divided by thenumber of active zoomed image areas, as shown in the following table:

Number Effective NTSC Frame Effective of Active Zoomed Rate per ZoomedImage PAL Frame Rate per Image Areas (ZUPs) Area Zoomed Image Area 1 30fps   25 fps 2 15 fps 12.5 fps 3 10 fps 8.33 fps

As previously described, one or more of zoomed image areas 1210, 1212and 1214 may be manipulated as a virtual pan-tilt-zoom (“PTZ”) camerausing standard camera control protocol commands, and/or protocolextensions.

In the preceding example square pixel units have been used to describezoomed image areas having 212×120 resolution. It will be understood thatthis resolution may be changed to rectangular pixel format. Furthermore,it will be understood that any given number of one or more zoomed imageareas may be simultaneously displayed with a primary image, includingmore than three zoomed image areas. Furthermore, it will be understoodthat zoomed image areas may be alternatively displayed in a space abovea primary image, and/or simultaneously in spaces above and below aprimary image (e.g., with appropriate size adjustment). Furthermore oneor more secondary outputs may be utilized to provide a higher resolutionpresentation to one of the selected 212×120 images.

Examples 1-5 describe and illustrate just a few of the many multipleresolution image combinations that may be realized using the disclosedsystems and methods, with it being understood that other combinations ofimage resolutions, image frame rates, numbers of displayed image areas,display formats (e.g., zoomed or unzoomed), display devices, etc. may beimplemented as desirable or needed for a given application using thedisclosed systems and methods.

Various embodiments of the disclosed systems and methods have beenillustrated and described herein with respect to CCTV technology.However, it will be understood that the disclosed systems and methodsmay be implemented with any video system technology and associatedcomponents in which one or more benefits of the disclosed systems andmethods may be realized or suitably implemented. Furthermore, thedisclosed systems and methods may be advantageously implemented in anyvideo system environment in which one or more benefits of the disclosedsystems and methods may be realized or suitably implemented. Examples ofsuch environments include, but are not limited to, video surveillanceenvironments, television studio environments, televised sports eventenvironments, military targeting or imaging environments, stored imageanalysis environments, etc. Furthermore, it will be understood thatalthough described herein in relation to square pixels, images of thedisclosed systems and methods may be alternatively scaled, displayedand/or processed using rectangular pixels.

While the invention may be adaptable to various modifications andalternative forms, specific embodiments have been shown by way ofexample and described herein. However, it should be understood that theinvention is not intended to be limited to the particular formsdisclosed. Rather, the invention is to cover all modifications,equivalents, and alternatives falling within the spirit and scope of theinvention as defined by the appended claims. Moreover, the differentaspects of the disclosed systems and methods may be utilized in variouscombinations and/or independently. Thus the invention is not limited toonly those combinations shown herein, but rather may include othercombinations.

1. A method of providing multiple images for transmission across ananalog interface, comprising: receiving at least one digital image datainput stream, said digital image data input stream containing digitalimage information; creating at least two digital image data streams fromsaid at least one digital data input stream, each of said at least twodigital image data streams comprising at least a portion of said digitalimage information; merging said at least two digital image data streamsinto a common digital image data output stream; converting said commondigital image data output stream into an analog image output stream; andproviding said analog output image stream for transmission across saidanalog interface; wherein said at least one analog interface is a videoanalog transmission interface; and wherein said step of creating atleast two digital image data streams comprises at least one of thefollowing steps performed prior to said step of merging said at leasttwo digital image data streams: segmenting an image frame of said atleast one digital image data input stream into multiple segments thateach correspond to one of said at least two digital image data streams,and inserting alignment data into one or more of said multiple segments;or extracting at least one portion of an image frame of said at leastone digital image data input stream to form a windowed partial imagecorresponding to one of said at least two digital image data streams,providing multiple possible available frame rates for each of said atleast two digital image data streams, and adapting a frame rate of atleast one of said at least two digital image data streams to one of saidmultiple possible available frame rates to match the bandwidthcharacteristics of said analog interface to allow the transmission of adesired amount of image information within the bandwidth of said analoginterface; or a combination thereof.
 2. The method of claim 1, whereinsaid method comprises extracting at least one portion of an image frameof said at least one digital image data input stream to form a windowedpartial image corresponding to one of said at least two digital imagedata streams, providing multiple possible available frame rates for eachof said at least two digital image data streams, and adapting a framerate of at least one of said at least two digital image data streams toone of said multiple possible available frame rates to match thebandwidth characteristics of said analog interface to allow thetransmission of a desired amount of image information within thebandwidth of said analog interface; wherein one of said at least twodigital image data streams comprises a first image having a firstresolution; wherein another of said at least two digital image datastreams comprises a second image having a second resolution; and whereinat least one of: said first and second resolutions are different, orsaid first image comprises a different portion of said digital imagedata input stream than said second image, or a combination thereof. 3.The method of claim 1, wherein one of said at least two digital imagedata streams is created to comprise a first image having a firstresolution and a first frame rate; wherein another of said at least twodigital image data streams is created to comprise a second image havinga second resolution and a second frame rate; wherein said firstresolution is higher than said second resolution, and said first framerate is lower than said second frame rate; and wherein said methodfurther comprises selecting said first and second frame rates so thatsaid first image and said second image are transmitted simultaneouslytogether across said analog interface as said analog output image streamwithout exceeding the maximum frame transmission rate capacity of saidinterface.
 4. The method of claim 2, wherein said at least one digitalimage data input stream comprises a digital video signal received from adigital video source; and wherein said method further comprisesproviding said analog output image stream as an analog video signal fortransmission across said analog interface; and further comprisingreceiving said analog video signal from across said analog interface,said analog video signal comprising said first and second images; anddisplaying said first and second images of said analog video signal onan analog display device.
 5. The method of claim 1, wherein said methodcomprises segmenting an image frame of said at least one digital imagedata input stream into multiple segments that each correspond to one ofsaid at least two digital image data streams, and inserting alignmentdata into one or more of said multiple segments
 4. 6. The method ofclaim 4, further comprising: creating a third digital image data streamfrom said at least one digital data input stream, said third digitalimage data stream comprising at least a portion of said digital imageinformation and having a third resolution; converting said third digitalimage data stream into a second analog image output stream; andproviding said second analog output image stream as a second analogvideo signal for transmission across a second analog interface;receiving said second analog video signal from across said second analoginterface, said analog video signal comprising said third image; anddisplaying said third image of said second analog video signal on asecond analog display device; wherein at least one of: said thirdresolution is different from said first and second resolutions, or saidthird image comprises a different portion of said digital image data, ora combination thereof.
 7. The method of claim 6, wherein said creatingfurther comprises extracting a window area from said original imageinformation and scaling said extracted window area to create said thirdimage as a zoomed image; and further comprising controlling a value ofsaid scaling in real time based at least in part on a command signal, orcontrolling the position of said extracted window area relative to saidoriginal image in real time based at least in part on a command signal,or a combination thereof.
 8. The method of claim 4, wherein said firstand second resolutions are different and wherein said method furthercomprises displaying said first image at said first resolution whilesimultaneously displaying said second image at said second resolution.9. The method of claim 5, wherein said creating further comprises usingscaling to create said first image as a zoomed image prior to said stepof merging; wherein said second image is not a zoomed image; and whereinsaid step of displaying comprises displaying said zoomed first image onan analog display device while simultaneously displaying said secondunzoomed image on said same analog display device.
 10. The method ofclaim 9, further comprising controlling a value of said scaling in realtime based at least in part on a command signal.
 11. The method of claim4, wherein said digital image information comprises an original image;wherein said step of creating said first image comprises extracting awindow area from said original image information to create said firstimage prior to said step of merging; and wherein said step of displayingcomprises displaying said first image on an analog display device whilesimultaneously displaying said second image on said same analog displaydevice.
 12. The method of claim 11, further comprising controlling theposition of said extracted window area relative to said original imagein real time based at least in part on a command signal.
 13. The methodof claim 4, wherein said digital image information comprises an originalimage; wherein said step of creating said first image comprisesextracting a window area from said original image information and thenupscaling said window area to create said first image as a zoomed imageprior to said step of merging; and wherein said step of creating furthercomprises downscaling said original image information to create saidsecond image; and wherein said step of displaying comprises displayingsaid zoomed first image on an analog display device while simultaneouslydisplaying said downscaled second image on said same analog displaydevice, said second image being downscaled such that it does not occupythe full analog display space of said analog display device, and saidfirst image being displayed on at least a portion of said analog displaydevice that is not occupied by said second image.
 14. The method ofclaim 1, wherein said analog interface is a video analog transmissioninterface having a limited transmission capacity that is insufficient totransmit said digital information of said digital image data inputstream; and wherein said method further comprises reducing the framerate of at least one of said at least two digital image data streamsbelow native frame rate to allow the transmission of said analog outputimage stream across said analog interface.
 15. The method of claim 13,further comprising at least one of controlling the position of saidextracted window area relative to said original image in real time basedat least in part on a command signal; controlling a value of saidupscaling of said extracted window area in real time based at least inpart on a command signal; or a combination thereof.
 16. The method ofclaim 4, wherein said at least two digital image data streams comprisesat least four digital image data streams; wherein a first one of said atleast four digital image data streams comprises a first image having afirst resolution; wherein a second one of said at least two digitalimage data streams comprises a second image having a second resolution;wherein a third one of said at least four digital image data streamscomprises a third image having a third resolution; and wherein a fourthone of said at least four digital image data streams comprises a thirdimage having a third resolution; wherein each of said at least fourimages either has a resolution that is different from each of said otherat least four images, or comprises a different portion of said digitalimage data input stream than each of said other of said at least fourimages, or a combination thereof; wherein said analog video signalcomprises said first, second, third and fourth images; wherein saiddigital image information comprises an original high resolution image,and wherein said step of creating said first, second and third imagescomprises extracting respective first, second and third window areasfrom said original image and then upscaling said first, second and thirdwindow areas to create said first, second and third images as zoomedimages prior to said step of merging; wherein said step of creatingfurther comprises downscaling said original image information to createsaid fourth image; and and wherein said step of displaying comprisesdisplaying said zoomed first, second, and third zoomed images on ananalog display device while simultaneously displaying said downscaledfourth image on said same analog display device, said fourth image beingdownscaled such that it does not occupy the full analog display space ofsaid analog display device, and said first, second and third imagesbeing displayed on at least a portion of said analog display device thatis not occupied by said fourth image.
 17. The method of claim 16,further comprising at least one of controlling the position of at leastone of said first, second, or third extracted window areas relative tosaid original image in real time based at least in part on a commandsignal; controlling a value of said upscaling of at lest one of saidfirst, second or third extracted window areas in real time based atleast in part on a command signal, or a combination thereof.
 18. Amethod of providing multiple images for transmission across an analoginterface, comprising: receiving at least one digital image data inputstream, said digital image data input stream containing digital imageinformation; creating at least two digital image data streams from saidat least one digital data input stream, each of said at least twodigital image data streams comprising at least a portion of said digitalimage information, and wherein a first one of said at least two digitalimage data streams comprises a first image having a first resolution,and wherein a second one of said at least two digital image data streamscomprises a second image having a second resolution; converting a firstone of said at least two digital image data streams into a first analogimage output stream, and converting a second one of said at least twodigital image data streams into a second analog image output stream;communicating said first analog output image stream across a firstanalog interface, and communicating said second analog output imagestream across a second analog interface; receiving said first analogoutput image stream from across said first analog interface, said firstanalog output image stream comprising said first image; receiving saidsecond analog output image stream from across said second analoginterface, said second analog output image stream comprising said secondimage; displaying said first image of said first analog output imagestream on a first analog display device; and displaying said secondimage of said second analog output image stream on a second analogdisplay device; wherein said step of creating at least two digital imagedata streams comprises the following step performed prior to said stepsof converting: extracting at least one portion of an image frame of saidat least one digital image data input stream to form a windowed partialimage corresponding to one of said at least two digital image datastreams, providing multiple possible available frame rates for each ofsaid at least two digital image data streams, and adapting a frame rateof at least one of said at least two digital image data streams to oneof said multiple possible available frame rates to match the bandwidthcharacteristics of said first or second analog interface to allow thetransmission of a desired amount of image information within thebandwidth of said first or second analog interface; and wherein each ofsaid at least first and second analog interfaces is a video analogtransmission interface.
 19. The method of claim 18, wherein said firstand second resolutions are different, or wherein said first imagecomprises a different portion of said digital image data input streamthan said second image, or a combination thereof.
 20. The method ofclaim 18, wherein said at least one digital image data input streamcomprises a digital video signal received from a digital video source;and wherein said method further comprises providing said first andsecond analog output image streams as first and second analog videosignals for transmission across said respective first and second analoginterfaces.
 21. A method of processing digital image data, comprising:providing said digital image data, said digital image data comprising afull image; processing said digital image data in a first processingoperation to create first processed image data comprising a first image;processing said digital image data in a second processing operation tocreate second processed image data comprising a second image; mergingsaid first and second processed image data into a common merged datastream; converting said merged data stream into a composite analog videostream containing said first and second images; and providing saidcomposite analog video stream for communication across an analoginterface; wherein at least one of: said first processed image data hasan image resolution that is different from an image resolution of saidsecond processed image data, or said first processed image datacomprises a different portion of said digital image data than saidsecond processed image data or a combination thereof; and wherein saidsteps of processing said digital image data in said first and secondprocessing operations each comprises at least one of the followingsteps: segmenting an image frame of said digital image data intomultiple segments that each correspond to one of said first or secondprocessed image data, and inserting alignment data into one or more ofsaid multiple segments; or extracting at least one portion of an imageframe of said digital image data to form a windowed partial imagecorresponding to one of said first or second processed image data,providing multiple possible available frame rates for each of said firstand second processed image data, and adapting a frame rate of at leastone of said first and second processed image data to one of saidmultiple possible available frame rates to match the bandwidthcharacteristics of said analog interface to allow the transmission of adesired amount of image information within the bandwidth of said analoginterface; or a combination thereof; and wherein said at least oneanalog interface is a video analog transmission interface.
 22. Themethod of claim 21, further comprising controlling at least one of saidfirst or second processing operations in real time based at least inpart on a command signal.
 23. The method of claim 21, furthercomprising: receiving said composite analog video stream containing saidfirst and second images from across said interface; and displaying saidfirst and second images contained in said composite analog video streamon a common analog display device.
 24. The method of claim 23, whereinsaid method comprises extracting at least one portion of an image frameof said digital image data to form a windowed partial imagecorresponding to one of said first or second processed image data,providing multiple possible available frame rates for each of said firstand second processed image data, and adapting a frame rate of at leastone of said first and second processed image data to one of saidmultiple possible available frame rates to match the bandwidthcharacteristics of said analog interface to allow the transmission of adesired amount of image information within the bandwidth of said analoginterface; and wherein said first processing operation comprises adownscaling operation; and wherein said second processing operationcomprises an image windowing operation, an image scaling operation, or acombination thereof.
 25. The method of claim 24, further comprisingcontrolling at least one of said first or second processing operationsin real time based at least in part on a command signal.
 26. The methodof claim 23, wherein said method comprises extracting at least oneportion of an image frame of said digital image data to form a windowedpartial image corresponding to one of said first or second processedimage data, providing multiple possible available frame rates for eachof said first and second processed image data, and adapting a frame rateof at least one of said processed image data to one of said multiplepossible available frame rates to match the bandwidth characteristics ofsaid analog interface to allow the transmission of a desired amount ofimage information within the bandwidth of said analog interface; andwherein said first processed image data comprises a downscaled fullimage and said second processed image data comprises a windowed image, ascaled image, or a combination thereof.
 27. The method of claim 26,further comprising: processing said digital image data in one or moreadditional processing operations to create one or more additionalprocessed image data comprising a third image; merging said one or moreadditional processed image data into said common merged data stream;converting said merged data stream into said composite analog videostream containing said first, second and one or more additional images;providing said composite analog video stream for communication acrosssaid analog interface; receiving said composite analog video streamcontaining said first, second and one or more additional images fromacross said interface; and displaying said first, second and one or moreadditional images contained in said composite analog video stream on acommon analog display device; wherein said second processing operationcomprises an image windowing operation and an image upscaling operation;wherein said second processed image data comprises an extracted andupscaled windowed image; wherein said one or more additional processingoperations comprises an image windowing operation and an image upscalingoperation; wherein said one or more additional processed image datacomprises an extracted and upscaled windowed image; and wherein at leastone of: said one or more additional processed image data has an imageresolution that is different from an image resolution of said first andsecond processed image data, or said one or more additional processedimage data comprises a different portion of said digital image data thansaid first and second processed image data, or a combination thereof.28. The method of claim 26, wherein said step of displaying comprisesdisplaying said downscaled first image on an analog display device whilesimultaneously displaying said windowed or scaled second image on saidsame analog display device, said first image being downscaled such thatit does not occupy the full analog display space of said analog displaydevice, and said second image being displayed in at least a portion ofsaid full analog display space not occupied by said first image.
 29. Themethod of claim 27, wherein said step of displaying comprises displayingsaid downscaled first image on an analog display device whilesimultaneously displaying said extracted and upscaled windowed secondimage and said extracted and upscaled windowed one or more additionalimages on said same analog display device, said first image beingdownscaled such that it does not occupy the full analog display space ofsaid analog display device, and said second image and said one or moreadditional images being displayed in at least a portion of said fullanalog display space not occupied by said first image.
 30. The method ofclaim 26, further comprising: processing said digital image data in oneor more additional processing operations to create one or moreadditional processed image data comprising a third image; providing saidone or more additional processed image data in a second data stream;converting said second data stream into a second composite analog videostream containing said one or more additional images; providing saidsecond composite analog video stream for communication across a secondanalog interface; receiving said second composite analog video streamcontaining said one or more additional images from across saidinterface; and displaying said one or more additional images containedin said second composite analog video stream on a second analog displaydevice.
 31. The method of claim 30, wherein said one or more additionalprocessing operations comprises an image windowing operation and animage upscaling operation; wherein said one or more additional processedimage data comprises an extracted and upscaled windowed image.
 32. Themethod of claim 31, wherein said method comprises segmenting an imageframe of said digital image data into multiple segments that eachcorrespond to one of said first or second processed image data, andinserting alignment data into one or more of said multiple segments. 33.Image creation circuitry configured to receive at least one digitalimage data input stream containing digital information, said imagecreation circuitry comprising: first image processing circuitryconfigured to create at least two digital image data streams from saidat least one digital data input stream, each of said at least twodigital image data streams comprising at least a portion of said digitalimage information, and said image processing circuitry configured toperform at least one of the following: segment an image frame of said atleast one digital image data input stream into multiple segments thateach correspond to one of said at least two digital image data streams,and insert alignment data into one or more of said multiple segments, orextract at least one portion of an image frame of said at least onedigital image data input stream to form a windowed partial imagecorresponding to one of said at least two digital image data streams,provide multiple possible available frame rates for each of said atleast two digital image data streams, and adapt a frame rate of at leastone of said at least two digital image data streams to one of saidmultiple possible available frame rates to match the bandwidthcharacteristics of a given analog interface to allow the transmission ofa desired amount of image information within the bandwidth of saidanalog interface; or a combination thereof; second image processingcircuitry configured to merge said at least two digital image datastreams into a common digital image data output stream; and conversioncircuitry coupled to said image processing circuitry and configured to:receive said common digital image data output stream from said imageprocessing circuitry, convert said common digital image data outputstream into an analog image output stream, and provide said analogoutput image stream for transmission across a video analog transmissioninterface.
 34. The image creation circuitry of claim 33, wherein saidfirst image processing circuitry is configured to extract at least oneportion of an image frame of said at least one digital image data inputstream to form a windowed partial image corresponding to one of said atleast two digital image data streams, provide multiple possibleavailable frame rates for each of said at least two digital image datastreams, and adapt a frame rate of at least one of said at least twodigital image data streams to one of said multiple possible availableframe rates to match the bandwidth characteristics of a given analoginterface to allow the transmission of a desired amount of imageinformation within the bandwidth of said analog interface; wherein saidat least one digital image data input stream comprises a digital videosignal received from a digital video source; and wherein said conversioncircuitry is further configured to provide said analog output imagestream as an analog video signal for transmission across said analoginterface; and wherein one of said at least two digital image datastreams comprises a first image having a first resolution; whereinanother of said at least two digital image data streams comprises asecond image having a second resolution; and wherein at least one of:said first and second resolutions are different, or said first imagecomprises a different portion of said digital image data input streamthan said second image, or a combination thereof.
 35. The image creationcircuitry of claim 33, wherein said image processing circuitry isconfigured to segment an image frame of said at least one digital imagedata input stream into multiple segments that each correspond to one ofsaid at least two digital image data streams, and insert alignment datainto one or more of said multiple segments.
 36. An image creation anddisplay system comprising said first and second image creation circuitryof claim 34; and further comprising an analog display device coupled tosaid analog interface and configured to receive said analog video signalfrom across said analog interface and to simultaneously display saidfirst and second images of said analog video signal.
 37. The imagecreation and display system of claim 36, wherein: said first imageprocessing circuitry is further configured to: create a third digitalimage data stream from said at least one digital data input stream, saidthird digital image data stream comprising at least a portion of saiddigital image information and having a third resolution; and saidconversion circuitry is further configured to: receive said thirddigital image data stream from said first image processing circuitry,convert said third digital image data stream into a second analog imageoutput stream, and provide said second analog output image stream as asecond analog video signal for transmission across a second analoginterface; and wherein said image creation and display system furthercomprises a second analog display device coupled to said second analoginterface and configured to: receive said second analog video signalfrom across said second analog interface, said second analog videosignal comprising said third image, and display said third image of saidsecond analog video signal; and wherein at least one of: said thirdresolution is different from said first and second resolutions, or saidthird image comprises a different portion of said digital image data, ora combination thereof.
 38. The image creation and display system ofclaim 37, wherein said first image processing circuitry is furtherconfigured to: extract a window area from said original imageinformation and to scale said extracted window area to create said thirdimage as a zoomed image; and control a value of said scaling in realtime based at least in part on a received command signal, or control theposition of said extracted window area relative to said original imagein real time based at least in part on a received command signal, or acombination thereof.
 39. The image creation circuitry of claim 36,wherein one of said at least two digital image data streams comprises afirst image having a first resolution and a first frame rate; whereinanother of said at least two digital image data streams comprises asecond image having a second resolution and a second frame rate; whereinsaid first resolution is higher than said second resolution, and saidfirst frame rate is lower than said second frame rate, said first andsecond frame rates being selected so that said first image and saidsecond image are transmitted simultaneously together across said analoginterface as said analog output image stream without exceeding themaximum frame transmission rate capacity of said interface.
 40. Theimage creation and display system of claim 36, wherein said first imageprocessing circuitry is further configured to use scaling to create saidfirst image as a zoomed image prior to said merging; wherein said secondimage is not a zoomed image; and wherein said analog display device isfurther configured to display said zoomed first image whilesimultaneously displaying said second unzoomed image.
 41. The imagecreation and display system of claim 36, wherein said image processingcircuitry is further configured to control a value of said scaling inreal time based at least in part on a received command signal.
 42. Theimage creation and display system of claim 36, wherein said digitalimage information comprises an original image; wherein said first imageprocessing circuitry is further configured to create said first image byextracting a window area from said original image information to createsaid first image prior to said merging; and wherein said analog displaydevice is further configured to display said first image whilesimultaneously displaying said second image.
 43. The image creation anddisplay system of claim 36, wherein said first image processingcircuitry is further configured to control a position of said extractedwindow area relative to said original image in real time based at leastin part on a received command signal.
 44. The image creation and displaysystem of claim 36, wherein said digital image information comprises anoriginal image; wherein said first image processing circuitry is furtherconfigured to create said first image by: extracting a window area fromsaid original image information and then upscaling said window area tocreate said first image as a zoomed image prior to said merging, anddownscaling said original image information to create said second image;and wherein said analog display device is further configured to displaysaid zoomed first image while simultaneously displaying said downscaledsecond image, said second image being downscaled such that it does notoccupy the full analog display space of said analog display device, saidfirst image being displayed on at least a portion of said analog displaydevice that is not occupied by said second image.
 45. The image creationand display system of claim 33, wherein said analog interface is a videoanalog transmission interface having a limited transmission capacitythat is insufficient to transmit said digital information of saiddigital image data input stream; and wherein said multiple resolutionimage creation circuitry is configured to reduce the frame rate of atleast one of said at least two digital image data streams below nativeframe rate to allow the transmission of said analog output image streamacross said analog interface.
 46. The image creation and display systemof claim 44, wherein said first image processing circuitry is furtherconfigured to: control a position of said extracted window area relativeto said original image in real time based at least in part on a receivedcommand signal; control a value of said upscaling of said extractedwindow area in real time based at least in part on a command signal; ora combination thereof.
 47. The image creation and display system ofclaim 36, wherein said at least two digital image data streams compriseat least four digital image data streams; wherein a first one of said atleast four digital image data streams comprises a first image having afirst resolution; wherein a second one of said at least two digitalimage data streams comprises a second image having a second resolution;wherein a third one of said at least four digital image data streamscomprises a third image having a third resolution; and wherein a fourthone of said at least four digital image data streams comprises a thirdimage having a third resolution; wherein each of said at least fourimages either has a resolution that is different from each of said otherat least four images, or comprises a different portion of said digitalimage data input stream than each of said other of said at least fourimages, or a combination thereof; wherein said analog video signalcomprises said first, second, third and fourth images; wherein saiddigital image information comprises an original high resolution image;and wherein said first image processing circuitry is further configuredto: create said first, second and third images by extracting respectivefirst, second and third window areas from said original image and thenupscaling said first, second and third window areas to create saidfirst, second and third images as zoomed images prior to said merging,downscale said original image information to create said fourth image;and wherein said analog display device is further configured to displaysaid zoomed first, second, and third zoomed images while simultaneouslydisplaying said downscaled fourth image, said fourth image beingdownscaled such that it does not occupy the full analog display space ofsaid analog display device, and said first, second and third imagesbeing displayed on at least a portion of said analog display device thatis not occupied by said fourth image.
 48. The image creation and displaysystem of claim 47, wherein said first image processing circuitry isfurther configured to: control the position of at least one of saidfirst, second, or third extracted window areas relative to said originalimage in real time based at least in part on a command signal; orcontrol a value of said upscaling of at lest one of said first, secondor third extracted window areas in real time based at least in part on acommand signal, or a combination thereof.
 49. A multiple resolutionimage creation and display system, comprising: multi-resolution imageprocessing circuitry configured to: receive at least one digital imagedata input stream, said digital image data input stream containingdigital image information, create at least two digital image datastreams from said at least one digital data input stream, each of saidat least two digital image data streams comprising at least a portion ofsaid digital image information, and wherein a first one of said at leasttwo digital image data streams comprises a first image having a firstresolution, and wherein a second one of said at least two digital imagedata streams comprises a second image having a second resolution,wherein said multi-resolution image processing circuitry is configuredto create said at least two digital image data streams from said atleast one digital data input stream by extracting at least one portionof an image frame of said at least one digital image data input streamto form a windowed partial image corresponding to one of said at leasttwo digital image data streams, providing multiple possible availableframe rates for each of said at least two digital image data streams,and adapting a frame rate of at least one of said at least two digitalimage data streams to one of said multiple possible available framerates to match the bandwidth characteristics of a given analog interfaceto allow the transmission of a desired amount of image informationwithin the bandwidth of said analog interface; conversion circuitrycoupled to said multi-resolution image processing circuitry andconfigured to: convert a first one of said at least two digital imagedata streams into a first analog image output stream, and convert asecond one of said at least two digital image data streams into a secondanalog image output stream, communicate said first analog output imagestream across a first analog interface, and communicate said secondanalog output image stream across a second analog interface; firstdisplay device coupled to said first analog interface and configured toreceive said first analog output image stream comprising said firstimage from across said first analog interface, and to display said firstimage; second display device coupled to said second analog interface andconfigured to receive said second analog output image stream comprisingsaid second image from across said second analog interface, and todisplay said second image; wherein each of said first and second analoginterfaces is a video analog transmission interface.
 50. The multipleresolution image creation and display system of claim 49, wherein saidfirst and second resolutions are different, or wherein said first imagecomprises a different portion of said digital image data input streamthan said second image, or a combination thereof.
 51. The multipleresolution image creation and display system of claim 49, wherein saidat least one digital image data input stream comprises a digital videosignal received from a digital video source; and wherein said conversioncircuitry is further configured to provide said first and second analogoutput image streams as first and second analog video signals fortransmission across said respective first and second analog interfaces.52. Multiple resolution image creation circuitry for processing digitalimage data comprising a fall image, said multiple resolution imagecreation circuitry comprising: multi-resolution image processingcircuitry configured to: process said digital image data in a firstprocessing operation to create first processed image data comprising afirst image, process said digital image data in a second processingoperation to create second processed image data comprising a secondimage, merge said first and second processed image data into a commonmerged data stream, wherein said multi-resolution image processingcircuitry is configured to process said digital image data in said firstand second processing operations to create each of said first and secondprocessed image data from said digital image data by extracting at leastone portion of an image frame of said digital image data to form awindowed partial image corresponding to one of said first and secondimages, providing multiple possible available frame rates for each ofsaid first and second processed image data, and adapting a frame rate ofat least one of said processed image data to one of said multiplepossible available frame rates to match the bandwidth characteristics ofa given analog interface to allow the transmission of a desired amountof image information within the bandwidth of said analog interface; andconversion circuitry configured to: convert said merged data stream intoa composite analog video stream containing said first and second images,and provide said composite analog video stream for communication acrossan analog interface; wherein at least one of: said first processed imagedata has an image resolution that is different from an image resolutionof said second processed image data, or said first processed image datacomprises a different portion of said digital image data than saidsecond processed image data, or a combination thereof.
 53. The multipleresolution image creation circuitry of claim 52, wherein saidmulti-resolution image processing circuitry is further configured tocontrol at least one of said first or second processing operations inreal time based at least in part on a command signal.
 54. A multipleresolution image creation and display system comprising said multipleresolution image creation circuitry of claim 52, and further comprising:an analog display device coupled to said analog interface and configuredto: receive said composite analog video stream containing said first andsecond images from across said interface; and simultaneously displaysaid first and second images contained in said composite analog videostream.
 55. The multiple resolution image creation and display system ofclaim 54, wherein said first processing operation comprises adownscaling operation; and wherein said second processing operationcomprises an image windowing operation, an image scaling operation, or acombination thereof.
 56. The multiple resolution image creation anddisplay system of claim 55, wherein said multi-resolution imageprocessing circuitry is further configured to control at least one ofsaid first or second processing operations in real time based at leastin part on a received command signal.
 57. The multiple resolution imagecreation and display system of claim 54, wherein said first processedimage data comprises a downscaled full image and said second processedimage data comprises a windowed image, a scaled image, or a combinationthereof.
 58. The multiple resolution image creation and display systemof claim 57, wherein: said multi-resolution image processing circuitryis further configured to: process said digital image data in one or moreadditional processing operations to create one or more additionalprocessed image data comprising a third image, merge said one or moreadditional processed image data into said common merged data stream;wherein said conversion circuitry is further configure to: convert saidmerged data stream into said composite analog video stream containingsaid first, second and one or more additional images, provide saidcomposite analog video stream for communication across said analoginterface; wherein said analog display device is further configured to:receive said composite analog video stream containing said first, secondand one or more additional images from across said interface, andsimultaneously display said first, second and one or more additionalimages contained in said composite analog video stream; wherein saidsecond processing operation comprises an image windowing operation andan image upscaling operation; wherein said second processed image datacomprises an extracted and upscaled windowed image; wherein said one ormore additional processing operations comprise an image windowingoperation and an image upscaling operation; wherein said one or moreadditional processed image data comprises an extracted and upscaledwindowed image; and wherein at least one of: said one or more additionalprocessed image data has an image resolution that is different from animage resolution of said first and second processed image data, or saidone or more additional processed image data comprises a differentportion of said digital image data than said first and second processedimage data, or a combination thereof.
 59. The multiple resolution imagecreation and display system of claim 57, wherein said analog displaydevice is further configured to display said downscaled first imagewhile simultaneously displaying said windowed or scaled second image,said first image being downscaled such that it does not occupy the fullanalog display space of said analog display device, and said secondimage being displayed in at least a portion of said full analog displayspace not occupied by said first image.
 60. The multiple resolutionimage creation and display system of claim 58, wherein said analogdisplay device is further configured to display said downscaled firstimage while simultaneously displaying said extracted and upscaledwindowed second image and said extracted and upscaled windowed one ormore additional images, said first image being downscaled such that itdoes not occupy the full analog display space of said analog displaydevice, and said second image and said one or more additional imagesbeing displayed in at least a portion of said full analog display spacenot occupied by said first image.
 61. The multiple resolution imagecreation and display system of claim 57, wherein: said multi-resolutionimage processing circuitry is further configured to: process saiddigital image data in one or more additional processing operations tocreate one or more additional processed image data comprising a thirdimage, provide said one or more additional processed image data in asecond data stream; wherein said conversion circuitry is furtherconfigured to: convert said second data stream into a second compositeanalog video stream containing said one or more additional images,provide said second composite analog video stream for communicationacross a second analog interface; and wherein said multiple resolutionimage creation and display system further comprises a second analogdisplay device coupled to said second analog interface and configuredto: receive said second composite analog video stream containing saidone or more additional images from across said second analog interface,and display said one or more additional images contained in said secondcomposite analog video stream.
 62. The multiple resolution imagecreation and display system of claim 61, wherein said one or moreadditional processing operations comprise an image windowing operationand an image upscaling operation; wherein said one or more additionalprocessed image data comprises an extracted and upscaled windowed image.63. The multiple resolution image creation and display system of claim62, wherein said multi-resolution image processing circuitry is furtherconfigured to control said one or more additional processing operationsin real time based at least in part on a received command signal.
 64. Animage processing and display system comprising: image creation circuitrycoupled to at least one analog display device by at least one analoginterface, wherein said at least one analog interface is a video analogtransmission interface having a limited transmission capacity that isinsufficient to transmit a given image signal, and wherein said imagecreation circuitry is configured to reduce a frame rate of said givenimage signal below a native frame rate to allow the transmission of saidgiven image signal.
 65. The image processing and display system of claim64, wherein said image creation circuitry comprises image processingcircuitry that comprises at least one window circuitry component, atleast one image scaler circuitry component, and at least one framebuffer circuitry component.
 66. The image processing and display systemof claim 64, further comprising at least two analog devices, each ofsaid at least two analog display devices being coupled to said imagecreation circuitry by a separate respective analog interface.
 67. Theimage processing and display system of claim 65, further comprising acommand circuitry coupled to said image processing circuitry configuredto provide one or more command signals to control at least one of saidat least one window circuitry component, at least one image scalercircuitry component, or a combination thereof.
 68. The image processingand display system of claim 67, wherein said command signals comprisecamera control commands.
 69. A method of providing multiple images fortransmission across a digital or analog interface, comprising: receivingat least one digital image data input stream, said digital image datainput stream containing digital image information; creating at least twodigital image data streams from said at least one digital data inputstream, each of said at least two digital image data streams comprisingat least a portion of said digital image information; merging said atleast two digital image data streams into a common digital image dataoutput stream; providing said common digital image data output streamfor transmission across said digital interface, or converting saidcommon digital image data output stream into an analog image outputstream and providing said analog output image stream for transmissionacross an analog interface; wherein said step of creating at least twodigital image data streams comprises at least one of the following stepsperformed prior to said step of merging said at least two digital imagedata streams: segmenting an image frame of said at least one digitalimage data input stream into multiple segments that each correspond toone of said at least two digital image data streams, and insertingalignment data into one or more of said multiple segments; or extractingat least one portion of an image frame of said at least one digitalimage data input stream to form a windowed partial image correspondingto one of said at least two digital image data streams, providingmultiple possible available frame rates for each of said at least twodigital image data streams, and varying a frame rate of at least one ofsaid at least two digital image data streams relative to the frame rateof another of said at least two digital image data streams; or acombination thereof.
 70. The method of claim 69, further comprisingreceiving said common digital image data output stream from across saiddigital interface, said common digital image data output streamcomprising said first and second images; and storing said first andsecond images, displaying said first and second images on a displaydevice, or a combination thereof.
 71. Image creation circuitryconfigured to receive at least one digital image data input streamcontaining digital information, said image creation circuitry comprisingimage processing circuitry configured to: create at least two digitalimage data streams from said at least one digital data input stream,each of said at least two digital image data streams comprising at leasta portion of said digital image information; merge said at least twodigital image data streams into a common digital image data outputstream; and provide said common digital image data output stream fortransmission across a digital interface, or convert said common digitalimage data output stream into an analog image output stream and providesaid analog output image stream for transmission across an analoginterface; wherein said image processing circuitry is configured toperform at least one of the following: segment an image frame of said atleast one digital image data input stream into multiple segments thateach correspond to one of said at least two digital image data streams,and insert alignment data into one or more of said multiple segments, orextract at least one portion of an image frame of said at least onedigital image data input stream to form a windowed partial imagecorresponding to one of said at least two digital image data streams,provide multiple possible available frame rates for each of said atleast two digital image data streams, and vary a frame rate of at leastone of said at least two digital image data streams relative to theframe rate of another of said at least two digital image data streams;or a combination thereof.
 72. An image creation and display systemcomprising said image creation circuitry of claim 71 and furthercomprising: a digital processing device coupled to said digitalinterface and configured to receive said common digital image dataoutput stream from across said digital interface and to store saidcommon digital image data output stream, to display one or more imagescontained in said common digital image data output stream, or acombination thereof.